Record code-trace canary fidelity gates

runs/frontier-code-trace-v0/README.md

@@ -1,18 +1,38 @@
 # Frontier code-trace campaign handoff
 
-本目录已经准备好无 GPU 的 data preflight、512→16 参数化映射、prefill-only 转换和 `max_model_len` 显式适配。当前没有启动或探测 `dash1`–`dash4`。
+Phase A code prefill+decode 已进入 61min real matrix。data/profile、
+max-length、Frontier rho calibration 和 TP2/TP4 paired canary 均已完成；
+第一批 TP4 三个 load 与 TP2 low-rho diagnostic 正在 dash1–dash4 并行
+运行。code prefill-only 的独立 sim calibration 也已完成。
 
 完整设计与 gate 见 [`experiment-card.md`](experiment-card.md)。
 
-## 当前已知阻塞
+## 当前资产与下一步
 
-本机 `/home/gahow/ali-trace/trace-glm5.1-formatted/` 不存在。仓库历史记录的远端路径是：
+- development window：0513 `[3480,7140)`，61min；
+- held-out window：0529 `[2640,6240)`，只在 development 判据冻结后使用；
+- profile：`profiles/profile-v6-code-longctx/`，覆盖 TP1/2/4 和 131072
+  KV context；
+- full paired inputs（CPFS）：
+  `runs/frontier-code-trace-v0/inputs/full-r0p{0002,0004,0008,0016}-v1/`；
+- compact provenance：`results/calibration-summary.json`、
+  `results/prefill-only-calibration-summary.json`、
+  `results/canary-analysis-tp{2,4}-v*.json` 和
+  `results/paired-input-manifests/`；
+- 当前 A4 wave 1：TP4 `rho={0.0002,0.0008,0.0016}` trial 1，以及
+  TP2 `rho=0.0002` trial 1 diagnostic；
+- TP4 canary 的 TTFT/E2E、prefix hit 与 decode batch 通过；TP2
+  TTFT p90 低估 32.1%，因此 TP2 其余 cell 暂不扩展；
+- prefill-only：TP2 已冻结 `rho={0.0004,0.0008,0.0016}`；
+  TP4 到 `0.0032` 仍亚临界，需追加更高 rho 后冻结 near-knee。
+
+source trace 的远端位置是：
 
 ```text
 /home/admin/cpfs/wjh/ali-trace/trace-glm5.1-formatted/
 ```
 
-有机器后先确认用户给出的 `~/ali-trace/...` 是否解析到同一目录，再运行以下命令。
+以下命令保留为从 source 重新构建时的复现入口。
 
 ## 1. 审计所有 1h+ code source
 
@@ -33,7 +53,12 @@ max_model_len_recommendation != null
 selected.selected_window_stats.max_model_len_coverage[推荐值].coverage = 1.0
 ```
 
-旧记录预计 source block size 为 512、推荐 max model len 为 131072，但禁止把这两个值写死为实验事实。
+全量审计已确认 source block size=512；development source window 若 100%
+覆盖需要 262144，但正式 server cap 以 session-sampled paired cell 的实际
+`ISL+OSL max` 向上对齐，不能把 full-window 262144 无条件套到低 rho cell。
+审计会单独记录并排除 `input_length<=0` 或 `output_length<=0` 的 source
+行；这些行只有在 raw trace 同样显示 zero usage/empty response 时才按
+“未发生模型执行”处理，不能无记录过滤。
 
 ## 2. 物化稳定窗口
 
@@ -86,23 +111,33 @@ python3 runs/frontier-s3-real-v0/remap_hash_blocks.py \
 现有 real runner 新增了三个显式环境变量，chat 默认行为不变：
 
 ```bash
-MAX_MODEL_LEN=131072 \
+MAX_MODEL_LEN=ACTUAL_CELL_MAX_ROUNDED_UP \
 TRACE_INPUT_ROOT=/absolute/path/to/materialized/code-cell \
 ALLOW_SYNTHETIC_PROMPTS=true \
 OUTPUT_ROOT=/absolute/path/to/new/output \
 bash runs/frontier-s3-real-v0/run_full_real.sh RHO_LABEL tp4_mns16 1 PORT
 ```
 
-- `MAX_MODEL_LEN` 必须等于 manifest 推荐值；
+- `MAX_MODEL_LEN` 必须覆盖 manifest 中该 paired cell 的实际最大请求；
 - `TRACE_INPUT_ROOT` 内必须有 `real_requests.jsonl` 和 `manifest.json`；
 - synthetic prompt 默认拒绝，只有在 experiment card 明确降级 claim 后才设为 `true`；
 - runner 会在启动前扫描 paired requests，若任何 `ISL+OSL` 超 cap 立即失败。
 
-正式 full job 前，先按 experiment card 的 G4 补 32k–128k attention profile，再做 TP4→TP2 的 p50/p99/max 单请求与 5min canary。
+长上下文 server 必须同时设置
+`VLLM_ALLOW_LONG_MAX_MODEL_LEN=1` 和
+`--hf-overrides '{"max_position_embeddings":MAX_MODEL_LEN}'`；runner 已在
+`ALLOW_LONG_CONTEXT_SERVER=true` 时自动处理。长上下文默认使用 host-local
+vLLM compile cache，并按 topology 复用 FlashInfer workspace；启动 compile
+不进入 workload latency。
 
 ## 6. decode-only
 
-当前 materializer 故意不提供 `decode_only` 选项。严格 decode-only 需要 initial-KV state，而不是把 prompt 改短。只有 real `DecodeBenchConnector`（或等价能力）与 Frontier initial-KV contract 都通过 G7 后，才创建 decode-only jobs。
+当前 materializer 故意不提供 `decode_only` 选项。已安装 vLLM 0.20.0
+包含 `DecodeBenchConnector`，但它在首次 admission 后同步填 dummy KV；
+fill time 必须与 KV-ready arrival 分离。Frontier `Request` 支持
+`num_processed_tokens`，当前 trace generator 尚未从 CSV 注入该值。
+只有 real 首步无 prefill、sim ledger 首步为 decode 的 C0 gate 通过后，
+才创建 strict decode-only jobs。
 
 ## 本地验证
 

runs/frontier-code-trace-v0/analyze_canary.py

@@ -0,0 +1,424 @@
+#!/usr/bin/env python3
+"""Analyze one paired 10-minute code-trace real/sim canary topology."""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import hashlib
+import json
+import math
+import re
+import statistics
+from collections import Counter
+from pathlib import Path
+from typing import Any
+
+
+METRICS = ("ttft", "tpot", "e2e")
+TPOT_MIN_OUTPUT_TOKENS = (2, 8, 32)
+SLO_TARGET_PASS_RATE = 0.95
+PROM_COUNTERS = ("vllm:prefix_cache_queries_total", "vllm:prefix_cache_hits_total")
+csv.field_size_limit(16 * 1024 * 1024)
+ITERATION_RE = re.compile(
+    r"Iteration.*?:\s+"
+    r"(?P<context_requests>\d+) context requests, "
+    r"(?P<context_tokens>\d+) context tokens, "
+    r"(?P<generation_requests>\d+) generation requests, "
+    r"(?P<generation_tokens>\d+) generation tokens"
+)
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--input-root", type=Path, required=True)
+    parser.add_argument("--sim-root", type=Path, required=True)
+    parser.add_argument("--real-root", type=Path, action="append", required=True)
+    parser.add_argument("--topology", required=True)
+    parser.add_argument("--output", type=Path, required=True)
+    return parser.parse_args()
+
+
+def sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as stream:
+        for chunk in iter(lambda: stream.read(1024 * 1024), b""):
+            digest.update(chunk)
+    return digest.hexdigest()
+
+
+def percentile(values: list[float], q: float) -> float:
+    ordered = sorted(values)
+    position = (len(ordered) - 1) * q
+    lower = math.floor(position)
+    upper = math.ceil(position)
+    if lower == upper:
+        return ordered[lower]
+    return ordered[lower] * (upper - position) + ordered[upper] * (position - lower)
+
+
+def distribution(values: list[float]) -> dict[str, float | int]:
+    if not values:
+        raise ValueError("empty distribution")
+    return {
+        "count": len(values),
+        "mean": statistics.fmean(values),
+        "p50": percentile(values, 0.5),
+        "p90": percentile(values, 0.9),
+        "p95": percentile(values, 0.95),
+        "p99": percentile(values, 0.99),
+        "max": max(values),
+    }
+
+
+def read_csv(path: Path) -> list[dict[str, str]]:
+    with path.open(newline="") as stream:
+        return list(csv.DictReader(stream))
+
+
+def find_one(root: Path, name: str) -> Path:
+    matches = list(root.glob(f"**/{name}"))
+    if len(matches) != 1:
+        raise ValueError(f"expected one {name} below {root}, found {matches}")
+    return matches[0]
+
+
+def prom_counter(path: Path, name: str) -> float:
+    values = []
+    with path.open() as stream:
+        for line in stream:
+            if line.startswith(name + "{") or line.startswith(name + " "):
+                values.append(float(line.rsplit(maxsplit=1)[1]))
+    if not values:
+        raise ValueError(f"{path}: missing Prometheus counter {name}")
+    return sum(values)
+
+
+def prefix_cache_delta(root: Path) -> dict[str, float]:
+    before = root / "metrics/before.prom"
+    after = root / "metrics/after.prom"
+    deltas = {
+        name: prom_counter(after, name) - prom_counter(before, name)
+        for name in PROM_COUNTERS
+    }
+    queries = deltas[PROM_COUNTERS[0]]
+    hits = deltas[PROM_COUNTERS[1]]
+    if queries <= 0 or hits < 0 or hits > queries:
+        raise ValueError(f"{root}: invalid prefix counter deltas {deltas}")
+    return {
+        "query_tokens": queries,
+        "hit_tokens": hits,
+        "hit_ratio": hits / queries,
+    }
+
+
+def real_decode_batch(root: Path) -> dict[str, Any]:
+    counts: Counter[int] = Counter()
+    mixed_steps = 0
+    for path in sorted(root.rglob("server.log")):
+        with path.open(errors="replace") as stream:
+            for line in stream:
+                match = ITERATION_RE.search(line)
+                if match is None:
+                    continue
+                context_requests = int(match.group("context_requests"))
+                generation_requests = int(match.group("generation_requests"))
+                generation_tokens = int(match.group("generation_tokens"))
+                if context_requests:
+                    mixed_steps += 1
+                    continue
+                if generation_requests and generation_tokens == generation_requests:
+                    counts[generation_requests] += 1
+
+    if not counts:
+        return {
+            "steps": 0,
+            "mixed_steps_excluded": mixed_steps,
+            "max": None,
+            "share_gt_1": None,
+            "histogram": {},
+        }
+    steps = sum(counts.values())
+    return {
+        "steps": steps,
+        "mixed_steps_excluded": mixed_steps,
+        "max": max(counts),
+        "share_gt_1": sum(value for key, value in counts.items() if key > 1) / steps,
+        "histogram": {str(key): value for key, value in sorted(counts.items())},
+    }
+
+
+def load_sim(root: Path, trace: list[dict[str, str]], trace_sha: str) -> dict[str, Any]:
+    manifest = json.loads((root / "manifest.json").read_text())
+    if manifest["trace_sha256"] != trace_sha:
+        raise ValueError(f"{root}: sim/input trace SHA mismatch")
+    rows = read_csv(find_one(root / "metrics", "request_metrics.csv"))
+    if len(rows) != len(trace):
+        raise ValueError(f"{root}: sim/input request count mismatch")
+    values = {
+        "ttft": [float(row["ttft"]) for row in rows],
+        "tpot": [float(row["tpot"]) for row in rows if row["tpot"].strip()],
+        "e2e": [float(row["request_e2e_time"]) for row in rows],
+        "waiting": [float(row["request_waiting_time_total"]) for row in rows],
+    }
+    completions = [
+        float(trace_row["arrived_at"]) + float(metric_row["request_e2e_time"]) / 1000
+        for trace_row, metric_row in zip(trace, rows)
+    ]
+    tail_index = max(range(len(completions)), key=completions.__getitem__)
+    last_arrival = max(float(row["arrived_at"]) for row in trace)
+    summary = json.loads((root / "summary.json").read_text())
+    slo_pass = []
+    for trace_row, metric_row in zip(trace, rows):
+        input_tokens = int(trace_row["num_prefill_tokens"])
+        ttft_threshold_ms = 1000 + 1000 * input_tokens / 8000
+        tpot = (
+            float(metric_row["tpot"])
+            if metric_row["tpot"].strip()
+            else None
+        )
+        slo_pass.append(
+            float(metric_row["ttft"]) <= ttft_threshold_ms
+            and (tpot is None or tpot <= 150)
+        )
+    return {
+        "values": values,
+        "tpot_by_min_output_tokens": {
+            str(threshold): [
+                float(metric_row["tpot"])
+                for trace_row, metric_row in zip(trace, rows)
+                if int(trace_row["num_decode_tokens"]) >= threshold
+                and metric_row["tpot"].strip()
+            ]
+            for threshold in TPOT_MIN_OUTPUT_TOKENS
+        },
+        "slo": {
+            "passed": sum(slo_pass),
+            "pass_rate": sum(slo_pass) / len(slo_pass),
+            "feasible": sum(slo_pass) / len(slo_pass) >= SLO_TARGET_PASS_RATE,
+        },
+        "summary": summary,
+        "drain": {
+            "last_arrival_s": last_arrival,
+            "last_completion_s": completions[tail_index],
+            "tail_after_last_arrival_s": completions[tail_index] - last_arrival,
+            "tail_driver": {
+                "request_index": tail_index,
+                "arrival_s": float(trace[tail_index]["arrived_at"]),
+                "arrival_before_cutoff_s": last_arrival
+                - float(trace[tail_index]["arrived_at"]),
+                "input_tokens": int(trace[tail_index]["num_prefill_tokens"]),
+                "output_tokens": int(trace[tail_index]["num_decode_tokens"]),
+                "waiting_ms": values["waiting"][tail_index],
+                "e2e_ms": values["e2e"][tail_index],
+            },
+        },
+    }
+
+
+def load_real(
+    root: Path,
+    input_manifest: dict[str, Any],
+    trace: list[dict[str, str]],
+) -> dict[str, Any]:
+    result_path = root / "results/result.json"
+    result = json.loads(result_path.read_text())
+    if result["contract"]["row_vector_sha256"] != input_manifest["paired_row_vector_sha256"]:
+        raise ValueError(f"{root}: real/input row digest mismatch")
+    requests = result["requests"]
+    if len(requests) != len(trace) or not all(row["success"] for row in requests):
+        raise ValueError(f"{root}: incomplete or failed real request vector")
+    for index, (request, trace_row) in enumerate(zip(requests, trace)):
+        observed = (int(request["input_tokens"]), int(request["requested_output_tokens"]))
+        expected = (
+            int(trace_row["num_prefill_tokens"]),
+            int(trace_row["num_decode_tokens"]),
+        )
+        if observed != expected:
+            raise ValueError(f"{root}: request {index} shape {observed} != {expected}")
+    values = {
+        metric: [
+            float(request[f"{metric}_ms"])
+            for request in requests
+            if request.get(f"{metric}_ms") is not None
+        ]
+        for metric in METRICS
+    }
+    completions = [
+        float(request["admitted_s"]) + float(request["e2e_ms"]) / 1000
+        for request in requests
+    ]
+    tail_index = max(range(len(completions)), key=completions.__getitem__)
+    last_arrival = max(float(request["scheduled_s"]) for request in requests)
+    return {
+        "root": str(root),
+        "result_sha256": sha256(result_path),
+        "values": values,
+        "tpot_by_min_output_tokens": {
+            str(threshold): [
+                float(request["tpot_ms"])
+                for request in requests
+                if int(request["requested_output_tokens"]) >= threshold
+                and request.get("tpot_ms") is not None
+            ]
+            for threshold in TPOT_MIN_OUTPUT_TOKENS
+        },
+        "slo": {
+            "passed": sum(bool(request["slo_pass"]) for request in requests),
+            "pass_rate": sum(bool(request["slo_pass"]) for request in requests)
+            / len(requests),
+            "feasible": sum(bool(request["slo_pass"]) for request in requests)
+            / len(requests)
+            >= SLO_TARGET_PASS_RATE,
+        },
+        "summary": result["summary"],
+        "prefix_cache": prefix_cache_delta(root),
+        "decode_batch": real_decode_batch(root),
+        "drain": {
+            "last_arrival_s": last_arrival,
+            "last_completion_s": completions[tail_index],
+            "tail_after_last_arrival_s": completions[tail_index] - last_arrival,
+            "tail_driver": {
+                "request_index": tail_index,
+                "arrival_s": float(requests[tail_index]["scheduled_s"]),
+                "arrival_before_cutoff_s": last_arrival
+                - float(requests[tail_index]["scheduled_s"]),
+                "input_tokens": int(requests[tail_index]["input_tokens"]),
+                "output_tokens": int(requests[tail_index]["requested_output_tokens"]),
+                "admission_lag_ms": float(requests[tail_index]["admission_lag_ms"]),
+                "e2e_ms": float(requests[tail_index]["e2e_ms"]),
+            },
+        },
+    }
+
+
+def main() -> None:
+    args = parse_args()
+    input_manifest = json.loads((args.input_root / "manifest.json").read_text())
+    trace_path = args.input_root / "frontier.csv"
+    trace = read_csv(trace_path)
+    if len(trace) != input_manifest["requests"]:
+        raise ValueError("input manifest/trace request count mismatch")
+    trace_sha = sha256(trace_path)
+    sim = load_sim(args.sim_root, trace, trace_sha)
+    reals = [load_real(root, input_manifest, trace) for root in args.real_root]
+    pooled = {
+        metric: [value for real in reals for value in real["values"][metric]]
+        for metric in METRICS
+    }
+    latency = {}
+    for metric in METRICS:
+        real_dist = distribution(pooled[metric])
+        real_per_trial = [
+            distribution(real["values"][metric]) for real in reals
+        ]
+        real_reference = {
+            statistic: statistics.fmean(
+                float(trial[statistic]) for trial in real_per_trial
+            )
+            for statistic in ("mean", "p50", "p90", "p95", "p99")
+        }
+        sim_dist = distribution(sim["values"][metric])
+        latency[metric] = {
+            "real": real_dist,
+            "real_trial_statistic_mean": real_reference,
+            "sim": sim_dist,
+            "relative_bias_percent": {
+                statistic: 100
+                * (float(sim_dist[statistic]) - real_reference[statistic])
+                / real_reference[statistic]
+                for statistic in ("mean", "p50", "p90", "p95", "p99")
+            },
+            "real_per_trial": real_per_trial,
+        }
+    tpot_sensitivity = {}
+    for threshold in TPOT_MIN_OUTPUT_TOKENS:
+        key = str(threshold)
+        real_per_trial = [
+            distribution(real["tpot_by_min_output_tokens"][key])
+            for real in reals
+        ]
+        real_values = [
+            value
+            for real in reals
+            for value in real["tpot_by_min_output_tokens"][key]
+        ]
+        sim_values = sim["tpot_by_min_output_tokens"][key]
+        real_dist = distribution(real_values)
+        real_reference = {
+            statistic: statistics.fmean(
+                float(trial[statistic]) for trial in real_per_trial
+            )
+            for statistic in ("mean", "p50", "p90", "p95", "p99")
+        }
+        sim_dist = distribution(sim_values)
+        tpot_sensitivity[key] = {
+            "real": real_dist,
+            "real_trial_statistic_mean": real_reference,
+            "real_per_trial": real_per_trial,
+            "sim": sim_dist,
+            "relative_bias_percent": {
+                statistic: 100
+                * (float(sim_dist[statistic]) - real_reference[statistic])
+                / real_reference[statistic]
+                for statistic in ("mean", "p50", "p90", "p95", "p99")
+            },
+        }
+    payload = {
+        "schema": "frontier-code-trace-canary-analysis-v1",
+        "topology": args.topology,
+        "requests_per_trial": len(trace),
+        "trials": len(reals),
+        "input": {
+            "manifest": str(args.input_root / "manifest.json"),
+            "paired_row_vector_sha256": input_manifest["paired_row_vector_sha256"],
+            "frontier_csv_sha256": trace_sha,
+        },
+        "latency_ms": latency,
+        "tpot_by_min_output_tokens": tpot_sensitivity,
+        "slo": {
+            "definition": {
+                "ttft_ms": "1000 + 1000 * input_tokens / 8000",
+                "tpot_ms": 150,
+                "target_pass_rate": SLO_TARGET_PASS_RATE,
+            },
+            "real_per_trial": [real["slo"] for real in reals],
+            "sim": sim["slo"],
+            "feasibility_flip": any(
+                real["slo"]["feasible"] != sim["slo"]["feasible"]
+                for real in reals
+            ),
+        },
+        "prefix_cache": {
+            "real_per_trial": [real["prefix_cache"] for real in reals],
+            "real_hit_ratio_mean": statistics.fmean(
+                real["prefix_cache"]["hit_ratio"] for real in reals
+            ),
+            "sim": sim["summary"]["prefix_cache"],
+        },
+        "drain": {
+            "interpretation": (
+                "Report the max-completion request explicitly; a response that "
+                "arrived well before the cutoff can create a long drain tail "
+                "without implying queue accumulation."
+            ),
+            "real_per_trial": [real["drain"] for real in reals],
+            "sim": sim["drain"],
+        },
+        "sim_decode_batch": sim["summary"]["decode_batch"],
+        "real_decode_batch_per_trial": [real["decode_batch"] for real in reals],
+        "real_artifacts": [
+            {
+                "root": real["root"],
+                "result_sha256": real["result_sha256"],
+            }
+            for real in reals
+        ],
+    }
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
+    print(json.dumps({"output": str(args.output), "topology": args.topology}))
+
+
+if __name__ == "__main__":
+    main()

runs/frontier-code-trace-v0/audit_code_trace.py

@@ -131,6 +131,9 @@ def choose_window(
 
 def scan_source(path: Path, args: argparse.Namespace) -> dict[str, Any]:
     rows = 0
+    source_rows = 0
+    invalid_zero_token_rows = 0
+    invalid_zero_token_examples: list[dict[str, Any]] = []
     first_timestamp = None
     last_timestamp = None
     previous_timestamp = None
@@ -145,6 +148,7 @@ def scan_source(path: Path, args: argparse.Namespace) -> dict[str, Any]:
     sampling_rows = 0
     schema_keys: Counter[str] = Counter()
     for line_number, row in iter_jsonl(path):
+        source_rows += 1
         missing = [
             key
             for key in ("timestamp", "input_length", "output_length")
@@ -153,6 +157,22 @@ def scan_source(path: Path, args: argparse.Namespace) -> dict[str, Any]:
         if missing:
             raise ValueError(f"{path}:{line_number}: missing required fields {missing}")
         timestamp = float(row["timestamp"])
+        input_tokens = int(row["input_length"])
+        output_tokens = int(row["output_length"])
+        schema_keys.update(row.keys())
+        if input_tokens <= 0 or output_tokens <= 0:
+            invalid_zero_token_rows += 1
+            if len(invalid_zero_token_examples) < 20:
+                invalid_zero_token_examples.append(
+                    {
+                        "line_number": line_number,
+                        "chat_id": row.get("chat_id"),
+                        "timestamp": timestamp,
+                        "input_length": input_tokens,
+                        "output_length": output_tokens,
+                    }
+                )
+            continue
         if first_timestamp is None:
             first_timestamp = timestamp
         if previous_timestamp is not None and timestamp < previous_timestamp:
@@ -169,10 +189,6 @@ def scan_source(path: Path, args: argparse.Namespace) -> dict[str, Any]:
                 max_gaps.get(previous_bin, 0.0),
                 timestamp - previous_timestamp,
             )
-        input_tokens = int(row["input_length"])
-        output_tokens = max(1, int(row["output_length"]))
-        if input_tokens <= 0:
-            raise ValueError(f"{path}:{line_number}: input_length must be positive")
         input_lengths.append(input_tokens)
         output_lengths.append(output_tokens)
         total_lengths.append(input_tokens + output_tokens)
@@ -186,7 +202,6 @@ def scan_source(path: Path, args: argparse.Namespace) -> dict[str, Any]:
             isinstance(row.get("prompt"), (str, list)) and bool(row.get("prompt"))
         )
         sampling_rows += int("sampling_u" in row)
-        schema_keys.update(row.keys())
         rows += 1
         previous_timestamp = timestamp
         last_timestamp = timestamp
@@ -205,6 +220,10 @@ def scan_source(path: Path, args: argparse.Namespace) -> dict[str, Any]:
     return {
         "source": str(path.resolve()),
         "rows": rows,
+        "source_rows": source_rows,
+        "invalid_zero_token_rows": invalid_zero_token_rows,
+        "invalid_zero_token_fraction": invalid_zero_token_rows / source_rows,
+        "invalid_zero_token_examples": invalid_zero_token_examples,
         "first_timestamp": first_timestamp,
         "last_timestamp": last_timestamp,
         "span_s": last_timestamp - first_timestamp,
@@ -253,7 +272,9 @@ def scan_window(source: Path, window: dict[str, Any]) -> dict[str, Any]:
         if timestamp >= end:
             break
         input_tokens = int(row["input_length"])
-        output_tokens = max(1, int(row["output_length"]))
+        output_tokens = int(row["output_length"])
+        if input_tokens <= 0 or output_tokens <= 0:
+            continue
         inputs.append(input_tokens)
         outputs.append(output_tokens)
         totals.append(input_tokens + output_tokens)

runs/frontier-code-trace-v0/check_longctx_profile_repeats.py

@@ -0,0 +1,79 @@
+#!/usr/bin/env python3
+"""Check fresh-process repeat stability for the code long-context grid."""
+
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--first", type=Path, nargs="+", required=True)
+    parser.add_argument("--second", type=Path, nargs="+", required=True)
+    parser.add_argument("--output", type=Path, required=True)
+    parser.add_argument("--max-relative-difference", type=float, default=0.05)
+    return parser.parse_args()
+
+
+def load(paths: list[Path]) -> dict[tuple[int, str], dict]:
+    rows: dict[tuple[int, str], dict] = {}
+    for path in paths:
+        payload = json.loads(path.read_text())
+        for row in payload["rows"]:
+            if row.get("error"):
+                raise ValueError(
+                    f"{path}: failed profile row {row['config']['batch_spec']}"
+                )
+            key = (
+                int(row["tensor_parallel_size"]),
+                str(row["config"]["batch_spec"]),
+            )
+            if key in rows:
+                raise ValueError(f"duplicate row {key}")
+            rows[key] = row
+    return rows
+
+
+def main() -> None:
+    args = parse_args()
+    first = load(args.first)
+    second = load(args.second)
+    if first.keys() != second.keys():
+        raise ValueError(
+            f"repeat key mismatch: first_only={sorted(first.keys()-second.keys())}, "
+            f"second_only={sorted(second.keys()-first.keys())}"
+        )
+    comparisons = []
+    for key in sorted(first):
+        left = float(first[key]["mean_time"])
+        right = float(second[key]["mean_time"])
+        relative = abs(left - right) / ((left + right) / 2)
+        comparisons.append(
+            {
+                "tp": key[0],
+                "batch_spec": key[1],
+                "first_mean_s": left,
+                "second_mean_s": right,
+                "relative_difference": relative,
+                "pass": relative <= args.max_relative_difference,
+            }
+        )
+    maximum = max(item["relative_difference"] for item in comparisons)
+    payload = {
+        "schema": "frontier-code-longctx-repeat-check-v1",
+        "threshold": args.max_relative_difference,
+        "maximum_relative_difference": maximum,
+        "status": "PASS" if maximum <= args.max_relative_difference else "FAIL",
+        "comparisons": comparisons,
+    }
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
+    print(json.dumps({"status": payload["status"], "max": maximum}))
+    if payload["status"] != "PASS":
+        raise SystemExit(1)
+
+
+if __name__ == "__main__":
+    main()

runs/frontier-code-trace-v0/experiment-card.md

@@ -1,6 +1,8 @@
 # EXP-CODE-TRACE：从 chat 1h trace 扩展到 code 与 phase-separated replay
 
-> **状态：READY_FOR_DATA PREFLIGHT，未启动 GPU。** 当前只完成本地适配与实验冻结；`dash1`--`dash4` 有整机空闲后按本文 gate 顺序推进。禁止使用 `dash0`。
+> **状态：RUNNING（Phase A code P+D）。** A0 数据/profile、A1
+> max-length smoke、A2 paired canary 与 A3 sim calibration 已完成；
+> A4 第一批 61min real jobs 正在 `dash1`--`dash4` 运行。禁止使用 `dash0`。
 
 ## 目标与成功定义
 
@@ -11,6 +13,16 @@
 
 本轮不是只看“能否跑完”。每个正式 cell 必须满足：同一 request vector、同一 arrival、同一 token shape、同一 prefix/initial-KV 合约、real 零失败、无持续 backlog，并同时报告 TTFT/TPOT/E2E、queue/batch、KV/prefix state 与 5min 分窗漂移。
 
+## 当前决策快照
+
+| 项目 | 当前结论 | 下一 gate |
+|---|---|---|
+| code P+D 数据 | 61min development window、long-context profile-v6、四个 paired full 输入已冻结 | 第一批 1h real 运行中 |
+| TP4 负载 | low/mid/near-knee=`rho 0.0002/0.0008/0.0016`；paired canary 的 TTFT/E2E、KV、batch 通过 | 三个 load 的 trial 1 运行中 |
+| TP2 负载 | `rho 0.0004` canary 暴露 TTFT p90 `-32.1%` bad case | 只跑 `rho 0.0002` 1h diagnostic，暂不铺满 |
+| code prefill-only | TP2 已冻结 `rho 0.0004/0.0008/0.0016`，`0.0032` 过载；TP4 到 `0.0032` 仍亚临界 | TP4 追加更高 rho 边界 |
+| strict decode-only | vLLM 0.20.0 有 `DecodeBenchConnector`；Frontier trace generator 尚不能注入 initial computed tokens | C0 contract canary，未进入正式结果 |
+
 ## 三种 workload mode 的冻结定义
 
 | Mode | 保留 | 改写 | 主指标 | 明确不声称 |
@@ -58,6 +70,7 @@ strict decode-only 必须同时具备：
 
 - timestamp 单调，存在 60–75min 连续稳定窗口；
 - `timestamp/input_length/output_length` 全行存在；
+- `input_length<=0` 或 `output_length<=0` 的行不进入 replay，但必须计数并保留样例。已抽查的两条 0→0 行在 raw trace 中同时满足 `usage.total_tokens=0`、`response_message={}`，属于未发生模型执行的 source request，不是 full-cache decode；
 - `hash_ids` 数量与某个 source block size 在全行严格满足
   `ceil(ISL/source_block_size)`；预计值 512，但以审计结果为准；
 - 记录 ISL/OSL/ISL+OSL 的 p50/p90/p95/p99/max、gap、request rate、prompt/sampling 字段覆盖。
@@ -66,10 +79,15 @@ strict decode-only 必须同时具备：
 
 ### G2：`max_model_len` data gate
 
-候选 cap 固定为 `40960/65536/98304/131072/262144`，选能 **100% 覆盖选中窗口 `ISL+OSL`** 的最小值。规则：
+source-window audit 先用 `40960/65536/98304/131072/262144` 给出完整
+窗口上界；真实 server 则使用能 **100% 覆盖该 rho 实际 paired requests
+的 `ISL+OSL`** 的最小 16-token 对齐值。规则：
 
-- 若 max≤131072，主路径使用 131072 或更小的审计推荐值；
-- 若存在 >131072 请求，不允许悄悄过滤。优先验证 262144；若 runtime 不可行，必须预注册过滤比例，并把 claim 改为“≤131072 子群”；
+- sampling 只按 session-coherent `sampling_u`，不得按 token length 过滤；
+- full source window 的 cap 用于记录 workload envelope，不强迫低 rho cell
+  为未被抽中的 outlier 预留 KV capacity；
+- 若某 paired cell max≤131072，使用 131072 或更小的对齐值；超过
+  131072 时按该 cell 实际 max 向上对齐，而不是直接跳到 262144；
 - Frontier 的 trace max tokens、predictor max tokens/request、vLLM `--max-model-len` 三处使用同一个 manifest 值。
 
 ### G3：prompt 与 prefix fidelity
@@ -86,7 +104,7 @@ strict decode-only 必须同时具备：
 
 现有 profile-v5 的 KV context 上界约 32k，对 code 不足。根据 development window 的 uncached-ISL 分布生成 profile-v6-code-longctx：
 
-- full chunk：`q8k`，context 至少覆盖 32k/48k/64k/80k/96k/112k/120k；
+- full chunk：`q8k`，context 至少覆盖 40k/56k/72k/88k/104k/120k/128k；
 - tail chunk：从真实 `ISL mod 8192` 的 p50/p90 选择 2–4k/4–6k 代表点；
 - TP1/2/4 分开采集，复测 `q1ks8k/q8ks32k` anchor；
 - 每点至少两次 fresh-process repeat；CV≤5%，anchor drift≤10%；
@@ -106,6 +124,14 @@ strict decode-only 必须同时具备：
 
 `max_model_len` 变大不等于每个请求都预占最大 KV，但会改变启动合法性与可表达的单请求上界；实际 KV 压力仍由并发 token state 决定。
 
+对本模型，`VLLM_ALLOW_LONG_MAX_MODEL_LEN=1` 只放宽 scheduler/config
+校验，不会扩展模型内部 RoPE cache；server 还必须显式传
+`--hf-overrides '{"max_position_embeddings":147456}'`。runner 对
+`MAX_MODEL_LEN>40960` 自动同时设置这两层。长上下文 job 默认使用
+host-local vLLM compile cache；FlashInfer workspace 按 topology 复用，
+避免每个 rho/trial 重编译同一组 fused-MoE kernels。两者只影响启动，
+不进入 replay latency。
+
 ### G6：每种 mode 独立标定 rho
 
 不能复用 P+D rho：
@@ -116,6 +142,12 @@ strict decode-only 必须同时具备：
 
 每种 workload×mode 选择 `low/mid/near-knee` 三点；正式点必须亚临界：全请求完成、无持续 backlog、drain tail≤10%、waiting p99 不单调随时间增长。跨 knee 点若运行，只作为 overload boundary，不支持“不发散”结论。
 
+`drain tail` 必须同时列出最后完成请求的 arrival、ISL、OSL、waiting
+和 E2E。早于 cutoff 到达但 OSL 很长的请求可以在最后 arrival 后继续
+decode；这属于 intrinsic response tail，不等价于 arrival cutoff 时仍有
+持续增长的 queue backlog。亚临界判断以 queue/waiting trajectory 和
+tail driver 分解共同决定，不能只用一个 drain 秒数。
+
 ### G7：strict decode-only capability gate
 
 先在 10min synthetic trace 上验证：
@@ -141,6 +173,11 @@ strict decode-only 必须同时具备：
 
 若某 topology 的 near-knee 过载，像现有 chat TP2/ρ0.01 一样排除，不为凑齐矩阵强跑。
 
+当前状态：A0–A3 完成。A4 第一批为 TP4
+`rho={0.0002,0.0008,0.0016}` trial 1，以及 TP2 `rho=0.0002`
+trial 1 diagnostic；其余 TP2 cell 等该 diagnostic 验证 canary bad case
+后再决定是否扩展。
+
 ### Phase B：chat/code prefill-only
 
 - 复用各自已物化 window，只把 OSL 改为 1；
@@ -159,6 +196,20 @@ strict decode-only 必须同时具备：
 
 decode profile/serving anchors 至少覆盖实际 batch p99。当前 whole-layer grid 只对少数 b≤8 有证据，且 b6 有长尾；在 MNS128 case 前必须补 b{1,2,4,8,16,32,64,128} 或实际访问 bucket，不能把 b8 常数外推到 b128。
 
+已安装 vLLM 0.20.0 的 `DecodeBenchConnector` 会在首次 schedule 时把
+`request.num_tokens-num_computed_tokens-1` 个 token 标为 external，
+同步向已分配的每层 KV blocks 写 dummy non-zero values，再从最后一个
+prompt token 开始 forward。因此它适合测大 context 下的 decode
+compute/scheduling，但 connector fill 发生在 client admission 之后：
+fill time 必须单独记账并从 KV-ready arrival/TPOT 口径中排除。
+dummy KV 也不提供真实 prompt-content 或 MoE-routing fidelity。
+
+Frontier commit `deadc4a3` 的 `Request` 已支持构造
+`num_processed_tokens`，但 `TraceReplayRequestGenerator` 不读取该列；
+因此 sim 侧仍需一个显式、可测试的 `initial_computed_tokens` trace
+contract。C0 必须同时证明 real 首个 model step 是 decode、Frontier
+首个 ledger stage 是 decode，之后才能解除 strict decode-only 的 BLOCKED。
+
 ## 指标与判据
 
 共同口径：
@@ -202,3 +253,94 @@ mode-specific：
 - code trace 来自 GLM5.1 业务，serving model 是 Qwen3-30B；若无原 prompt text，测试只能保持 shape/prefix 结构，不能证明内容相关 routing fidelity；
 - `max_model_len=128k/256k` 解决的是接入上界，不自动解决 32k 以上 profile 外推；
 - strict decode-only 只测 decode engine；完整 PD 分离还需要单独建模 prefill、KV transfer、backpressure 与 KV-ready arrival。
+
+## 执行记录（2026-07-23）
+
+- fleet probe：dash1–dash4 均为 8×H20；32 张卡 memory.used=0、
+  utilization=0、无 compute process、uncorrected ECC=0；
+- 两个 formatted trace 都严格满足 512-token source hash contract；
+- 0513：2,108,130 个有效请求、6090 个 zero-usage source 行；稳定
+  development window=`[3480,7140)`，61min、1,078,928 请求；
+- 0529：1,977,423 个有效请求、6031 个 zero-usage source 行；冻结为
+  held-out，稳定候选 window=`[2640,6240)`；
+- development window：ISL p50/p90/p99/max =
+  20,051/88,224/125,803/202,371；OSL p50/p90/p99/max =
+  78/758/6449/131,072；`ISL+OSL max=202,745`；
+- full-window 131072 coverage=99.399%，262144 coverage=100%。但
+  session sampling 的候选 `rho<=0.0032` 实际 max total=137,016，因此
+  primary server cap 将按最终 cell max 对齐，不为未抽中的 202k outlier
+  直接预留 262k；
+- source 无 Qwen-aligned prompt/token IDs。raw canonical prompt 使用 GLM
+  token contract，不能同时保持 Qwen token content 与 trace ISL；本 campaign
+  采用 synthetic Qwen tokens 保持 length/hash/prefix shape，并降级内容 claim。
+- selected rho=0.0032 中有 1355 个可检查 parent links：tail rewrite
+  p50/p90/p95/p99/max=1/1/4/57/169 个 source blocks，说明 coder
+  `parent_chat_id` 不等价于 append-only prompt。source hash 序列作为 prefix
+  truth；synthetic content block 生成后再计算 parent-sensitive runtime
+  identities，避免“相同内容块出现在不同前缀后”造成 Frontier false hit。
+- 远端 Qwen3-30B `config.json` 的原生 position limit 是 40960
+  (`rope_theta=1e6`,无 rope_scaling)。147456 profile smoke 在显式
+  `VLLM_ALLOW_LONG_MAX_MODEL_LEN=1` 下成功；该 override 只支持
+  performance/shape fidelity，不形成生成质量或模型长上下文正确性 claim，
+  并作为 provenance 中的显式实验变量。
+- profile-v6-code-longctx 覆盖 TP1/2/4、KV context 到 131072，
+  33 个 long-context rows；两次 fresh-process repeat 的最大相对差
+  4.648%，旧 anchor drift 最大 1.7%。attention profile SHA256 =
+  `fbcf7e1f95789a6f6d771e24d1fc60958b7daf04eb0db260d27869a19d71d550`。
+- TP4 `max_model_len=147456` smoke 已在 dash4 通过。server 日志同时确认
+  `max_model_len=147456` 与 `hf_overrides.max_position_embeddings=147456`；
+  20,051+78、119,702+68、136,774+242 三个 shape 均成功。对应
+  TTFT=853.29/12,699.04/3,820.55ms，TPOT=16.65/7.21/8.13ms。
+  最长请求的非单调 TTFT 来自 cold compile/cache state，因此这里只作为
+  runtime support gate，不作为 profile accuracy 数据。
+- calibration 全部使用同一 profile-v6 SHA。TP4 的 `rho=0.0016`
+  decode batch max=16、drain=21.07s，仍通过 10% 亚临界 gate；TP2 的
+  `rho=0.0016` waiting p50=332.97s、drain=976.86s，明确过载并排除。
+  完整 compact table 在 `results/calibration-summary.json`。
+- TP2/TP4 的 `rho={0.0002,0.0004,0.0008,0.0016}` 61min full paired
+  inputs 已在 CPFS 物化；每个 paired row digest 和 Frontier CSV SHA
+  均与 calibration input 逐项一致。manifest 副本在
+  `results/paired-input-manifests/`。最大一个目录约 901MiB，不把大型
+  token arrays 提交进 Git。
+- code prefill-only 的最大 calibration cache 已物化：
+  3477 requests、总 prefill 115,828,371 tokens、OSL 全为 1，
+  paired digest=`40865068e02414612ba1cd4595894e20e85e01fd34d73f8660185552d531ecea`。
+- 多 host 并发 server startup 暴露出 shared CPFS AOT cache 和每-job
+  FlashInfer JIT 的 apparatus cost。它发生在 readiness 前，不进入 TTFT；
+  runner commit `d5bb974` 改为长上下文默认使用 host-local vLLM cache，
+  并按 topology 复用 FlashInfer workspace。
+- 第一轮 paired real canary 的 3 次旧 client 运行都只在同一个
+  `106709+197` 请求失败，根因是 `return_token_ids` 把 100k+ prompt
+  vector 放进单条 SSE event，超过 aiohttp 默认 512KiB line limit。
+  commit `e1f2557` 把 exact client read buffer 提到 8MiB；700KiB
+  单-event runtime 对照和随后 TP4×2、TP2×1 的 53/53 replay 均通过。
+- TP4 canary 的 real-vs-sim prefix hit ratio =
+  `0.239908/0.239973`；pure-decode batch max 都为 4，
+  `share(b>1)=15.87%/15.69%`（real 两 trial）vs `16.35%`（sim）。
+  TTFT mean/p50/p90/p95/p99 bias =
+  `-8.3/-4.1/-8.8/-13.1/-6.3%`，E2E =
+  `+2.2/+6.1/+11.0/+1.4/-1.3%`。长 drain 的同一
+  `61976+21361` 请求 real=92.61/92.34s、sim=91.37s，不是 backlog。
+- TP4 若把 OSL=4 请求纳入 TPOT，mean/p99 bias 会被单个
+  `~213ms/token` 样本放大到 `-29.9%/-71.5%`；OSL≥8 后
+  mean/p50/p90/p95/p99 bias =
+  `+8.7/+8.2/+0.6/+13.4/+3.7%`。因此 raw TPOT 仍保留，但正式报告必须
+  同时给 OSL threshold sensitivity，不能把短输出的三段 inter-token
+  interval 当作稳定 decode service。
+- TP4 canary 两 trial 的 real SLO pass rate 都是 `50/53=94.34%`，
+  sim 为 `52/53=98.11%`，在 95% feasibility threshold 上发生翻转；
+  这由两个临界 TTFT 请求和上述 OSL=4 请求共同造成，作为明确 bad case
+  进入 1h 检验，不能被总体 latency gap 掩盖。
+- TP2 canary 的 cache/batch/drain 仍对齐，但 TTFT p90 bias=`-32.1%`，
+  E2E p90/p95=`-18.4%/-23.8%`。因此先只启动 low-rho 1h diagnostic，
+  不直接铺满 TP2 六个正式 jobs。
+- code prefill-only 已完成 10-cell Frontier calibration。TP2
+  `rho=0.0032` drain=1384.59s、waiting p50=751.01s，明确过载；
+  `0.0004/0.0008/0.0016` 冻结为 low/mid/near-knee。TP4 到
+  `rho=0.0032` 仍只有 9.09s drain，暂称 highest-tested，追加更高 rho
+  后才冻结 near-knee。compact table 在
+  `results/prefill-only-calibration-summary.json`。
+- 2026-07-23 18:37 UTC 启动 A4 wave 1：dash1=`TP4/rho0.0002/t1`、
+  dash3=`TP4/rho0.0008/t1`、dash4=`TP4/rho0.0016/t1`、
+  dash2=`TP2/rho0.0002/t1 diagnostic`；四台启动前再次确认 8×H20
+  memory/utilization=0、无 compute process、uncorrected ECC=0。

runs/frontier-code-trace-v0/prepare_code_window.py

@@ -63,6 +63,8 @@ def main() -> None:
                 continue
             if timestamp >= end:
                 break
+            if int(row["input_length"]) <= 0 or int(row["output_length"]) <= 0:
+                continue
             chat = row.get("chat_id", source_index)
             parent = row.get("parent_chat_id")
             has_parent = parent not in (None, "", -1, "-1")

runs/frontier-code-trace-v0/profiles/profile-v6-code-longctx/attention.csv

@@ -0,0 +1,277 @@
+time_stats.attn_input_reshape.min,time_stats.attn_input_reshape.max,time_stats.attn_input_reshape.mean,time_stats.attn_input_reshape.median,time_stats.attn_input_reshape.std,time_stats.attn_kv_cache_save.min,time_stats.attn_kv_cache_save.max,time_stats.attn_kv_cache_save.mean,time_stats.attn_kv_cache_save.median,time_stats.attn_kv_cache_save.std,time_stats.attn_prefill.min,time_stats.attn_prefill.max,time_stats.attn_prefill.mean,time_stats.attn_prefill.median,time_stats.attn_prefill.std,time_stats.attn_decode.min,time_stats.attn_decode.max,time_stats.attn_decode.mean,time_stats.attn_decode.median,time_stats.attn_decode.std,time_stats.attn_output_reshape.min,time_stats.attn_output_reshape.max,time_stats.attn_output_reshape.mean,time_stats.attn_output_reshape.median,time_stats.attn_output_reshape.std,n_embd,n_q_head,n_kv_head,block_size,num_tensor_parallel_workers,max_model_len,batch_size,prefill_chunk_size,kv_cache_size,is_prefill,attention_backend,is_mixed_batch,mode,seq_lens,total_tokens,max_seq_len,min_seq_len,avg_seq_len,equal_seq_len,seq_len_variance,seq_len_std,seq_len_cv,is_chunked_prefill_sample,chunk_start_token,chunk_end_token,total_prefill_tokens,profiling_precision,model_arch,quant_signature,measurement_type,is_true_mixed_batch,prefill_seq_lens,prefill_kv_cache_sizes,decode_kv_cache_sizes,num_prefill_seqs,num_decode_seqs,decode_batch_size,total_batch_size,total_decode_tokens,decode_avg_kv_cache_size,batch_composition_ratio,batch_spec,projection_policy
+0.0,0.0,0.0,0.0,0.0,0.01414399966597557,0.028863999992609024,0.019705599918961526,0.01771199982613325,0.005157200849836681,0.047968000173568726,0.07046400010585785,0.05810240097343922,0.05810240097343922,0.007477463486041561,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,64,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,False,0.0,64.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.04947200044989586,0.020412799902260303,0.01635199971497059,0.010107497379722417,0.046560000628232956,0.08323200047016144,0.05587520003318787,0.05587520003318787,0.011126758739503428,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,128,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,False,0.0,128.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.022207999601960182,0.017033600155264138,0.015312000177800655,0.002819235991970241,0.05104000121355057,0.07692799717187881,0.056396800279617305,0.056396800279617305,0.007481982178637539,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,256,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,False,0.0,256.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015072000212967396,0.022272000089287758,0.01706880023702979,0.01616000011563301,0.002460889579319197,0.06931199878454208,0.0838719978928566,0.07432000041007995,0.07432000041007995,0.004777766433175866,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,512,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,False,0.0,512.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.018592000007629395,0.028543999418616295,0.02095999978482723,0.019183999858796597,0.003198175496053494,0.12179200351238251,0.15408000349998474,0.1307712011039257,0.1307712011039257,0.00858807797538298,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.027775999158620834,0.03385600075125694,0.030131200328469276,0.029680000618100166,0.0021152558103575215,0.32678401470184326,0.3450239896774292,0.33396480381488797,0.33396480381488797,0.0045872424917606375,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04438399896025658,0.05084799975156784,0.046540799736976626,0.04531199857592583,0.002277905811237223,1.0959680080413818,1.1151360273361206,1.0999775886535645,1.0999775886535645,0.005694403246120485,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.078015998005867,0.08691199868917465,0.08114239946007729,0.08019199967384338,0.00292795706334475,4.070400238037109,4.113152027130127,4.087088012695312,4.087088012695312,0.013660567012509554,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016063999384641647,0.05167999863624573,0.022115200012922286,0.017583999782800674,0.010340094822340818,0.05196800082921982,0.09011200070381165,0.06328320093452933,0.06328320093452933,0.012557341255467452,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,64,448.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,True,448.0,512.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64s512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01583999954164028,0.026623999699950218,0.018927999772131443,0.017376000061631203,0.003514650316260619,0.06681600213050842,0.07993599772453308,0.0725280001759529,0.0725280001759529,0.004343502558613716,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,128,896.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,True,896.0,1024.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01648000068962574,0.030880000442266464,0.01945280022919178,0.017967999912798405,0.004096211183007485,0.1311360001564026,0.1546880006790161,0.13908160030841826,0.13908160030841826,0.007511906874366178,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,256,1792.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,True,1792.0,2048.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.017855999991297722,0.03558399900794029,0.020851199887692927,0.018559999763965607,0.005235911594130716,0.32950401306152344,0.350271999835968,0.33912960588932034,0.33912960588932034,0.006027400986663648,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,512,3584.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,True,3584.0,4096.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.019328000023961067,0.040608000010252,0.022790400311350822,0.020704000256955624,0.006113051965778337,1.1415679454803467,1.1518720388412476,1.144483208656311,1.144483208656311,0.0032332311374389127,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,1024,7168.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,True,7168.0,8192.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1ks8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015807999297976494,0.030688000842928886,0.019596799835562707,0.01774400006979704,0.004343384771033462,0.0,0.0,0.0,0.0,0.0,0.049056001007556915,0.07580800354480743,0.05948160067200661,0.05948160067200661,0.009031541471446955,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01603199914097786,0.02486399933695793,0.01923839971423149,0.018079999834299088,0.0032282528537266424,0.0,0.0,0.0,0.0,0.0,0.05142400041222572,0.07353600114583969,0.059328000620007516,0.059328000620007516,0.0073307807735143084,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016543999314308167,0.03977600112557411,0.021379199624061585,0.018511999398469925,0.006593576176246171,0.0,0.0,0.0,0.0,0.0,0.0488319993019104,0.06435199826955795,0.05479039996862411,0.05479039996862411,0.005672522998491864,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01635199971497059,0.02844800055027008,0.019267200119793416,0.017952000722289085,0.0035068687666949577,0.0,0.0,0.0,0.0,0.0,0.049855999648571014,0.07798399776220322,0.05986879989504815,0.05986879989504815,0.01043914754878828,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016383999958634377,0.026079999282956123,0.01923519968986511,0.017791999503970146,0.0032161974331284568,0.0,0.0,0.0,0.0,0.0,0.058111999183893204,0.1045759990811348,0.06708480007946492,0.06708480007946492,0.013479022462646494,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.04057599976658821,0.019100800156593323,0.015455999877303839,0.007512281243011577,0.0,0.0,0.0,0.0,0.0,0.05363199859857559,0.07782399654388428,0.06090559959411622,0.06090559959411622,0.007544620176348091,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014431999996304512,0.02191999927163124,0.016684799920767546,0.01563199982047081,0.0024293118621811216,0.0,0.0,0.0,0.0,0.0,0.0629120022058487,0.07891199737787247,0.06891520097851753,0.06891520097851753,0.005472695665695425,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014560000039637089,0.038943998515605927,0.018313600029796363,0.01561600062996149,0.007127270260115769,0.0,0.0,0.0,0.0,0.0,0.08675199747085571,0.10662399977445602,0.09391999915242194,0.09391999915242194,0.006988099589086635,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.054687999188899994,0.021439999900758268,0.01539199985563755,0.012052764849597775,0.0,0.0,0.0,0.0,0.0,0.13488000631332397,0.1528639942407608,0.1431359991431236,0.1431359991431236,0.005436271464033599,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.041919998824596405,0.01899839974939823,0.015343999955803156,0.007989843526623287,0.0,0.0,0.0,0.0,0.0,0.06176000088453293,0.08374399691820145,0.06747519969940186,0.06747519969940186,0.0066067747128778,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016256000846624374,0.11353600025177002,0.042761600017547606,0.028960000723600388,0.029104301538020762,0.0,0.0,0.0,0.0,0.0,0.09734400361776352,0.14422400295734406,0.11392960175871848,0.11392960175871848,0.013198594600417867,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.034143999218940735,0.018918400071561335,0.01643200032413006,0.005500943993080684,0.0,0.0,0.0,0.0,0.0,0.12918399274349213,0.15087999403476715,0.13807999789714814,0.13807999789714814,0.007658330538677587,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016063999384641647,0.03641600161790848,0.0198208000510931,0.01780799962580204,0.0057264128169845765,0.0,0.0,0.0,0.0,0.0,0.22099199891090393,0.23904000222682953,0.2293503984808922,0.2293503984808922,0.004861342907006028,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014751999638974667,0.035840000957250595,0.018908800091594458,0.015792000107467175,0.0064374817924757475,0.0,0.0,0.0,0.0,0.0,0.10134399682283401,0.12201599776744843,0.10896319895982742,0.10896319895982742,0.006336330809165179,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013856000266969204,0.03417599946260452,0.017846399918198586,0.014800000004470348,0.006495007539635255,0.0,0.0,0.0,0.0,0.0,0.13126400113105774,0.15561600029468536,0.1389280006289482,0.1389280006289482,0.008381472811075022,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014431999996304512,0.03519999980926514,0.019168000388890504,0.01600000075995922,0.005995522477654695,0.0,0.0,0.0,0.0,0.0,0.21728000044822693,0.2343679964542389,0.2231455981731415,0.2231455981731415,0.004720730646739123,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014047999866306782,0.03670400008559227,0.018441599886864425,0.015023999847471714,0.006596535162793127,0.0,0.0,0.0,0.0,0.0,0.39321601390838623,0.4524799883365631,0.4058080047369003,0.4058080047369003,0.01578349755088941,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014336000196635723,0.022143999114632607,0.016912000067532063,0.015168000012636185,0.0028156089295136347,0.0,0.0,0.0,0.0,0.0,0.15587200224399567,0.3079040050506592,0.17838079929351805,0.17838079929351805,0.04355865575265927,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.02112000063061714,0.015516800060868263,0.01488000014796853,0.001940870731593904,0.0,0.0,0.0,0.0,0.0,0.21587200462818146,0.23561599850654602,0.22250880002975468,0.22250880002975468,0.006181951170646666,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015552000142633915,0.039264000952243805,0.023609600123018028,0.02131200022995472,0.007236979625711548,0.0,0.0,0.0,0.0,0.0,0.408735990524292,0.470335990190506,0.4336863994598388,0.4336863994598388,0.01844662383160074,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.025407999753952026,0.016227199975401164,0.014960000291466713,0.0031617375441736185,0.0,0.0,0.0,0.0,0.0,0.7412800192832947,0.7627840042114258,0.7464000046253203,0.7464000046253203,0.006112167448837547,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01462399959564209,0.02812799997627735,0.020652799773961304,0.021359999664127827,0.004308706957613102,0.028383498565450627,0.039859687970646644,0.032492258074592426,0.032492258074592426,0.00453597266208597,0.029312501176103633,0.04116431058787463,0.03355574193327539,0.03355574193327539,0.004684436757701648,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,72,,,,,,,,False,,,64,BF16,generic,none,CUDA_EVENT,True,[64],[0],"[512, 512, 512, 512, 512, 512, 512, 512]",1,8,8,9,8,512.0,0.1111111111111111,q64_8q1s512,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.015552000142633915,0.034143999218940735,0.023171199765056372,0.024255999363958836,0.005908614918096971,0.03333159243114438,0.038935341782478095,0.03580428402241854,0.03580428402241854,0.002082270297044095,0.03633240903369937,0.04244065945634484,0.03902771507087562,0.03902771507087562,0.0022697354261490147,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,136,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,8,8,9,8,1024.0,0.1111111111111111,q128_8q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.02611199952661991,0.019635199941694735,0.018112000077962875,0.0038298050749257795,0.04189529417991216,0.057484239920526384,0.04744885718421094,0.04744885718421094,0.004779830748455743,0.051672703037266184,0.07089975418195164,0.05852234134479412,0.05852234134479412,0.005895334539786378,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,144,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,16,16,17,16,1024.0,0.058823529411764705,q128_16q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.029311999678611755,0.018662399891763926,0.01673599984496832,0.0044017112162725355,0.04322973959325901,0.05049827064705393,0.04535414343408448,0.04535414343408448,0.0022944156000240697,0.08733025617719538,0.10201372836398578,0.09162185574241775,0.09162185574241775,0.004635047631846023,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,272,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,16,16,17,16,2048.0,0.058823529411764705,q256_16q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014592000283300877,0.026367999613285065,0.016336000058799982,0.01515199989080429,0.003415353455946402,0.06031842775160765,0.06618323188375198,0.06274415549817247,0.06274415549817247,0.001985647289644255,0.14768157653992678,0.16204076249052324,0.15362064543185072,0.15362064543185072,0.004861590945216247,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,288,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,32,32,33,32,2048.0,0.030303030303030304,q256_32q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014751999638974667,0.02454400062561035,0.017167999967932702,0.016191999427974224,0.0028685016454498436,0.09128700688359712,0.09689150775996329,0.09360396051475776,0.09360396051475776,0.0013477542744916764,0.2740889887523892,0.2909164873209846,0.2810456356995366,0.2810456356995366,0.004046628526808561,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,544,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q512_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01881599985063076,0.03097599931061268,0.024598400108516216,0.024848000146448612,0.0037539437391565975,0.1035249255866932,0.10603132147437412,0.10399797220840973,0.10399797220840973,0.0007025109429056814,0.5652750707893444,0.5789606938874117,0.5678580377517648,0.5678580377517648,0.003835906384194897,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,576,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,64,64,65,64,4096.0,0.015384615384615385,q512_64q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018624000251293182,0.043487999588251114,0.024460799992084503,0.019600000232458115,0.008922081629781394,0.196169204945307,0.2076140047945799,0.2008444429250931,0.2008444429250931,0.00394350953292805,1.1196708009293268,1.1849940417370974,1.1463555573610091,1.1463555573610091,0.022508285530529783,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192]",1,64,64,65,64,8192.0,0.015384615384615385,q1k_64q1s8k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.027135999873280525,0.03667199984192848,0.02945920005440712,0.028256000019609928,0.0029446678408169553,0.37757279619664613,0.3898113624476372,0.38075520430942184,0.38075520430942184,0.0036600203831042254,0.2522831942132049,0.2604606493092597,0.25440958703617433,0.25440958703617433,0.0024455194930253126,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q2k_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.0435199998319149,0.049695998430252075,0.04502719938755036,0.04395199939608574,0.002035785660169308,1.1048984388245497,1.1189053886476108,1.1112371236754128,1.1112371236754128,0.0050220696724624985,0.1395495077239122,0.14131859607071193,0.14035008841031085,0.14035008841031085,0.0006342911944856293,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,16,16,17,16,4096.0,0.058823529411764705,q4k_16q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018783999606966972,0.024288000538945198,0.020627199858427047,0.019567999988794327,0.0019457052717059358,0.09455999732017517,0.12185599654912949,0.10207359939813614,0.10207359939813614,0.00753346544014261,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,2,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512]",1024,512,512,512.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.027295999228954315,0.034752000123262405,0.02943360023200512,0.028528000228106976,0.0023159160681123767,0.14416000247001648,0.15904000401496887,0.14979200065135959,0.14979200065135959,0.00480624675866005,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,4,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512]",2048,512,512,512.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.043455999344587326,0.05084799975156784,0.045500800386071204,0.04411200061440468,0.002490556062831049,0.24454399943351746,0.25865599513053894,0.2504959970712662,0.2504959970712662,0.003778494140301353,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512]",4096,512,512,512.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07673600316047668,0.08374399691820145,0.07970559895038605,0.07873599976301193,0.0022907976135004057,0.445248007774353,0.4758400022983551,0.45409599840641024,0.45409599840641024,0.009133230340383306,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512]",8192,512,512,512.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.043296001851558685,0.051072001457214355,0.044972800090909,0.04399999976158142,0.002451216824441962,0.6147199869155884,0.6290879845619202,0.6204927921295166,0.6204927921295166,0.004344846739788608,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,2,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048]",4096,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07737600058317184,0.09123200178146362,0.08094720020890236,0.07980800047516823,0.004065264798368611,1.1744320392608643,1.1887680292129517,1.178323209285736,1.178323209285736,0.004395876469319665,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,4,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048, 2048, 2048]",8192,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.03254399821162224,0.01810879958793521,0.01611199975013733,0.005058478889747962,0.0,0.0,0.0,0.0,0.0,0.06339199841022491,0.0841279998421669,0.07019519805908202,0.07019519805908202,0.006070126182471763,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.030271999537944794,0.018124799989163876,0.015711999498307705,0.004696400814632773,0.0,0.0,0.0,0.0,0.0,0.2739199995994568,0.2922239899635315,0.28281279802322384,0.28281279802322384,0.00633037978599564,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.022431999444961548,0.016844799742102623,0.015471999999135733,0.0027624803153841917,0.0,0.0,0.0,0.0,0.0,0.3909119963645935,0.4160960018634796,0.39996159672737125,0.39996159672737125,0.007037174636804898,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014240000396966934,0.035071998834609985,0.018764800019562246,0.015583999920636415,0.006305594700523817,0.0,0.0,0.0,0.0,0.0,0.7383679747581482,0.7597119808197021,0.7430047929286957,0.7430047929286957,0.006192645960911466,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.028991999104619026,0.019401599932461978,0.01665600063279271,0.005434953793780546,0.0,0.0,0.0,0.0,0.0,1.427008032798767,1.4517120122909546,1.4361984014511109,1.4361984014511109,0.008802755821028187,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.02844800055027008,0.01820160010829568,0.014944000169634819,0.004942002036909087,0.0,0.0,0.0,0.0,0.0,0.08361600339412689,0.11027199774980545,0.09058240056037903,0.09058240056037903,0.00872938604423622,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014112000353634357,0.023104000836610794,0.01648960020393133,0.015632000286132097,0.0027628828340349578,0.0,0.0,0.0,0.0,0.0,0.5063040256500244,0.5497599840164185,0.5168287932872773,0.5168287932872773,0.013557229606313293,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01616000011563301,0.029055999591946602,0.018780799955129622,0.017215999774634838,0.003794434947431941,0.0,0.0,0.0,0.0,0.0,0.7439360022544861,0.7719680070877075,0.7502080142498017,0.7502080142498017,0.00839205598262567,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015968000516295433,0.03046399913728237,0.018822400271892546,0.01726400014013052,0.004212536831927695,0.0,0.0,0.0,0.0,0.0,1.4256000518798828,1.449504017829895,1.4325888037681578,1.4325888037681578,0.0065448028108711165,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.027712000533938408,0.01633920017629862,0.014944000169634819,0.0038488220071983326,0.0,0.0,0.0,0.0,0.0,2.798719882965088,3.0184640884399414,2.8293471813201903,2.8293471813201903,0.06375662767704417,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.021407999098300934,0.01637439979240298,0.014992000069469213,0.0024463594774459265,0.0,0.0,0.0,0.0,0.0,0.09347199648618698,0.10473600029945374,0.09820479974150656,0.09820479974150656,0.004061668684999473,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014303999952971935,0.04790399968624115,0.020179199893027543,0.016447999514639378,0.009635010283506967,0.0,0.0,0.0,0.0,0.0,0.6221439838409424,0.6444799900054932,0.6277984082698822,0.6277984082698822,0.007234140179397069,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.03276799991726875,0.01921279989182949,0.015232000034302473,0.0065506148511265076,0.0,0.0,0.0,0.0,0.0,0.9111359715461731,0.9307839870452881,0.9151648044586183,0.9151648044586183,0.005528712062927265,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014175999909639359,0.02707199938595295,0.016512000095099212,0.014688000082969666,0.0038989080209321414,0.0,0.0,0.0,0.0,0.0,1.7645119428634644,1.7849279642105103,1.7684095859527589,1.7684095859527589,0.005876963340184193,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014240000396966934,0.029184000566601753,0.016825600154697896,0.014752000104635954,0.00457910514148248,0.0,0.0,0.0,0.0,0.0,3.4781761169433594,3.5388801097869873,3.490892815589905,3.490892815589905,0.01818910600692522,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014976000413298607,0.03139200061559677,0.01898880014196038,0.016848000697791576,0.005083715524393418,0.13752702814163098,0.14117629917511842,0.13848196486571557,0.13848196486571557,0.0009880564964713527,0.523336967410432,0.5372236808930884,0.5269708253945994,0.5269708253945994,0.00375988994658546,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,520,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,8,8,9,8,16384.0,0.1111111111111111,q512_8q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01833599992096424,0.02707199938595295,0.020995199866592883,0.01976000051945448,0.0028392656352050185,0.17817885890237703,0.18622915300900206,0.18109068484526028,0.18109068484526028,0.0027040006080457624,0.5449571488834201,0.5695788427872015,0.5538629212834322,0.5538629212834322,0.008270141985514729,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1040,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,16,16,17,16,16384.0,0.058823529411764705,q1k_16q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.02703999914228916,0.03308799862861633,0.028883199393749236,0.02759999968111515,0.002285758578932753,0.4688266550410815,0.48301665772804186,0.4727750380198454,0.4727750380198454,0.004396396389258256,1.0430453981052807,1.0746153117238624,1.051829759343436,1.051829759343436,0.009781101336187197,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,32,32,33,32,16384.0,0.030303030303030304,q2k_32q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.04368000105023384,0.059328000992536545,0.04809600040316582,0.04617599956691265,0.005158792915002645,1.3509461459747514,1.3759066409666496,1.3589365122072,1.3589365122072,0.008008877716183384,0.9213738861449998,0.9383974724214122,0.9268234851606594,0.9268234851606594,0.005462224239661061,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,16,16,17,16,32768.0,0.058823529411764705,q4k_16q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.026944000273942947,0.040511999279260635,0.030291200056672095,0.02817599941045046,0.004386386489305873,0.5071830964059985,0.5142792136001758,0.5096392092471257,0.5096392092471257,0.0022059272685869546,2.175632932188972,2.2060727208328075,2.186168772243963,2.186168772243963,0.009462633998570079,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,32,32,33,32,32768.0,0.030303030303030304,q2k_32q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018303999677300453,0.03017600066959858,0.02039040010422468,0.018943999893963337,0.0034612051983613614,0.21341429693945616,0.21552776300295579,0.21393496609766816,0.21393496609766816,0.0005736773262009222,4.617401495144284,4.663128147488988,4.628666619290515,4.628666619290515,0.012412001359412127,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,64,64,65,64,32768.0,0.015384615384615385,q1k_64q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.02364799939095974,0.01809599995613098,0.016352000646293163,0.0035481127058959038,0.04822399839758873,0.08566399663686752,0.05961279980838299,0.05961279980838299,0.011445665413968877,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,64,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,False,0.0,64.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014175999909639359,0.020864000543951988,0.01668160008266568,0.015664000064134598,0.0025769063833097584,0.049695998430252075,0.08057600259780884,0.05882879942655563,0.05882879942655563,0.009515126108519331,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,128,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,False,0.0,128.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.028704000636935234,0.017430400010198355,0.015056000091135502,0.004349294613335555,0.049855999648571014,0.07366400212049484,0.05459520071744919,0.05459520071744919,0.0069610920757925574,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,256,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,False,0.0,256.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014336000196635723,0.03161599859595299,0.017788799852132796,0.015711999963968992,0.005005385723318659,0.06102399900555611,0.08179199695587158,0.06650560013949873,0.06650560013949873,0.006947995595801105,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,512,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,False,0.0,512.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.04416000097990036,0.019200000166893005,0.015488000120967627,0.008561241323364038,0.08054400235414505,0.09196799993515015,0.08607039973139763,0.08607039973139763,0.004145329035483754,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.017855999991297722,0.023391999304294586,0.019667199812829494,0.018463999964296818,0.0022577669687832585,0.18729600310325623,0.20585599541664124,0.19546559900045393,0.19546559900045393,0.006339068824303663,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.026240000501275063,0.03446400165557861,0.028297600522637367,0.027088000439107418,0.0025148441522922374,0.5754240155220032,0.5889919996261597,0.5800191938877105,0.5800191938877105,0.003858869273829596,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04182400181889534,0.047200001776218414,0.043036799877882004,0.0423360001295805,0.0017073405772076728,2.063199996948242,2.0787200927734375,2.067151999473572,2.067151999473572,0.004959651271127835,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.05648000165820122,0.02270399993285537,0.017935999669134617,0.012377915150727689,0.049536000937223434,0.07196799665689468,0.056015999615192415,0.056015999615192415,0.0070476637552742884,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,64,448.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,True,448.0,512.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64s512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01408000010997057,0.021344000473618507,0.01611520005390048,0.014928000047802925,0.0025499884993961702,0.07072000205516815,0.2642880082130432,0.1588256008923054,0.1588256008923054,0.053220347086102376,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,128,896.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,True,896.0,1024.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01360000018030405,0.03014400042593479,0.016975999902933837,0.015056000091135502,0.004715159697364111,0.08393599838018417,0.11036799848079681,0.09160000011324881,0.09160000011324881,0.007911669434472792,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,256,1792.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,True,1792.0,2048.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.020767999812960625,0.016233599931001663,0.015392000321298838,0.002202615907995427,0.1998399943113327,0.22070400416851044,0.20855360180139543,0.20855360180139543,0.00689307200230667,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,512,3584.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,True,3584.0,4096.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.033440001308918,0.018684800155460833,0.016048000194132328,0.00553991005639174,0.6043199896812439,0.635807991027832,0.6126143991947173,0.6126143991947173,0.008745933953408096,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,1024,7168.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,True,7168.0,8192.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1ks8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013824000023305416,0.03359999880194664,0.017811199743300678,0.014944000169634819,0.005926140483565472,0.0,0.0,0.0,0.0,0.0,0.045471999794244766,0.07539200037717819,0.054758400097489356,0.054758400097489356,0.010253548506101549,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014047999866306782,0.02409599907696247,0.01641279999166727,0.01473599998280406,0.003347158638713987,0.0,0.0,0.0,0.0,0.0,0.0461760014295578,0.07529599964618683,0.05460800044238568,0.05460800044238568,0.009748937798340135,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.028416000306606293,0.018611199874430894,0.01598400017246604,0.005209875093863647,0.0,0.0,0.0,0.0,0.0,0.048128001391887665,0.07897599786520004,0.061353600397706036,0.061353600397706036,0.010488153655157845,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014112000353634357,0.025728000327944756,0.016092800162732603,0.01512000011280179,0.003300888123052605,0.0,0.0,0.0,0.0,0.0,0.04864000156521797,0.07648000121116638,0.05810560062527656,0.05810560062527656,0.009473544218404408,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.03551999852061272,0.018588799890130757,0.016064000315964222,0.006071057437992447,0.0,0.0,0.0,0.0,0.0,0.04822399839758873,0.07862400263547897,0.05660480037331582,0.05660480037331582,0.009565394213730401,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014303999952971935,0.028831999748945236,0.01699519995599985,0.015024000313133001,0.004285000744868188,0.0,0.0,0.0,0.0,0.0,0.04854400083422661,0.06719999760389328,0.05778240002691746,0.05778240002691746,0.006852125554679805,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.0144640002399683,0.024927999824285507,0.0161183999851346,0.01521599991247058,0.0029774808524673907,0.0,0.0,0.0,0.0,0.0,0.05379199981689453,0.08966399729251862,0.06270079985260964,0.06270079985260964,0.009983591277092696,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014431999996304512,0.03001599945127964,0.017648000083863736,0.01547200046479702,0.004585126309484848,0.0,0.0,0.0,0.0,0.0,0.061664000153541565,0.07692799717187881,0.06704320013523103,0.06704320013523103,0.005500191798490629,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014175999909639359,0.026367999613285065,0.016947199776768684,0.015039999969303608,0.0037951566103550205,0.0,0.0,0.0,0.0,0.0,0.08799999952316284,0.111455999314785,0.0964031994342804,0.0964031994342804,0.007397541615558088,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01369599997997284,0.021247999742627144,0.015359999984502793,0.014640000183135271,0.0020942770950814317,0.0,0.0,0.0,0.0,0.0,0.051711998879909515,0.07065600156784058,0.058054400235414506,0.058054400235414506,0.006633034815910025,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014208000153303146,0.0307839997112751,0.017148799914866685,0.015039999969303608,0.004882374686312284,0.0,0.0,0.0,0.0,0.0,0.061919998377561576,0.07843200117349625,0.06628479920327664,0.06628479920327664,0.004962852689801192,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013887999579310417,0.02969600073993206,0.017500799987465142,0.015232000034302473,0.00467273319705314,0.0,0.0,0.0,0.0,0.0,0.08819200098514557,0.11097600311040878,0.09493440166115762,0.09493440166115762,0.007509235042985577,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.022112000733613968,0.01751680001616478,0.016047999262809753,0.0030306621792915785,0.0,0.0,0.0,0.0,0.0,0.13065600395202637,0.15110400319099426,0.13857279866933822,0.13857279866933822,0.00750137841249771,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013919999822974205,0.04012800008058548,0.01892479993402958,0.01550400024279952,0.007459545267816961,0.0,0.0,0.0,0.0,0.0,0.06278400123119354,0.08259200304746628,0.0704512007534504,0.0704512007534504,0.005979055984382744,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014336000196635723,0.02236800082027912,0.016332800220698118,0.014928000047802925,0.002784044363186731,0.0,0.0,0.0,0.0,0.0,0.1003199964761734,0.1279360055923462,0.10921279862523078,0.10921279862523078,0.00862769617046716,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013919999822974205,0.0225600004196167,0.016128000058233737,0.014800000004470348,0.002958953332547708,0.0,0.0,0.0,0.0,0.0,0.13116799294948578,0.14812800288200378,0.13783999979496003,0.13783999979496003,0.005696384361148053,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014208000153303146,0.029983999207615852,0.017468800116330386,0.01508800033479929,0.0047379550962483065,0.0,0.0,0.0,0.0,0.0,0.217631995677948,0.2447360008955002,0.22715839892625808,0.22715839892625808,0.008831828221847138,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014047999866306782,0.02304000034928322,0.016512000095099212,0.014512000139802694,0.0035026774828624254,0.0,0.0,0.0,0.0,0.0,0.11020799726247787,0.12307199835777283,0.11600959971547126,0.11600959971547126,0.004667637266950902,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014112000353634357,0.042080000042915344,0.017510399967432023,0.014607999939471483,0.00823117883530167,0.0,0.0,0.0,0.0,0.0,0.15702399611473083,0.17587199807167053,0.16399359852075576,0.16399359852075576,0.006588299074676393,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013919999822974205,0.0208320003002882,0.015299199987202883,0.01462399959564209,0.001916768086505386,0.0,0.0,0.0,0.0,0.0,0.21779200434684753,0.2415360063314438,0.2260768011212349,0.2260768011212349,0.007251352080685236,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014015999622642994,0.021088000386953354,0.015619200188666582,0.01473599998280406,0.002013227452302141,0.0,0.0,0.0,0.0,0.0,0.3959999978542328,0.4152640104293823,0.40332479774951924,0.40332479774951924,0.006942401431914052,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014175999909639359,0.03379200026392937,0.020595200080424547,0.019600000232458115,0.006548754248881344,0.026192623739694512,0.040006882507168426,0.029155180178492036,0.029155180178492036,0.00408828748438241,0.024591375524545756,0.037561119537986146,0.027372820355088746,0.027372820355088746,0.0038383559348575944,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,72,,,,,,,,False,,,64,BF16,generic,none,CUDA_EVENT,True,[64],[0],"[512, 512, 512, 512, 512, 512, 512, 512]",1,8,8,9,8,512.0,0.1111111111111111,q64_8q1s512,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.013856000266969204,0.020896000787615776,0.015318400040268899,0.014431999996304512,0.0019640312960926966,0.029349018208693862,0.06236262941356679,0.03714313592014963,0.03714313592014963,0.009618313791872569,0.028826981462526914,0.06125337308649041,0.036482463672774496,0.036482463672774496,0.009447230957011863,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,136,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,8,8,9,8,1024.0,0.1111111111111111,q128_8q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.013856000266969204,0.032127998769283295,0.017286399938166143,0.014479999896138906,0.005536495446636193,0.03273085874558354,0.04394578491937082,0.03563837490653034,0.03563837490653034,0.003429601730256567,0.03488514202593899,0.04683821345079978,0.037984025407085474,0.037984025407085474,0.0036553316361902684,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,144,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,16,16,17,16,1024.0,0.058823529411764705,q128_16q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.013856000266969204,0.021503999829292297,0.015078400075435639,0.01425600005313754,0.002238435975478849,0.042100813549974185,0.051784144690147756,0.045378692890289625,0.045378692890289625,0.003184109940650555,0.05111518843748309,0.06287185663927425,0.05509490773570219,0.05509490773570219,0.0038658725544299132,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,272,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,16,16,17,16,2048.0,0.058823529411764705,q256_16q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.02687999978661537,0.017667199857532977,0.01566399959847331,0.003864811846387173,0.048475323773821306,0.05599956978723733,0.05123499252968345,0.05123499252968345,0.002427379820423941,0.08429268135885387,0.09737642835214408,0.0890914090616175,0.0890914090616175,0.0042209177332077005,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,288,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,32,32,33,32,2048.0,0.030303030303030304,q256_32q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01462399959564209,0.03359999880194664,0.019804799742996693,0.016032000072300434,0.006750519908145474,0.07121508474579985,0.07292308367136396,0.07194410845270183,0.07194410845270183,0.0005500065915943844,0.14760091249712767,0.15114092373010238,0.14911189243564577,0.14911189243564577,0.0011399477384397005,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,544,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q512_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.021856000646948814,0.016835200227797033,0.015263999812304974,0.00283854448975065,0.08146338272142935,0.08560865714384096,0.0834932625520734,0.0834932625520734,0.0012259635905347874,0.2782486219401307,0.29240733788179385,0.2851819365989658,0.2851819365989658,0.004187435731481665,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,576,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,64,64,65,64,4096.0,0.015384615384615385,q512_64q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.04495999962091446,0.02143679987639189,0.015856000129133463,0.009709987872683342,0.1194459208702178,0.12302524755181463,0.12053885718696096,0.12053885718696096,0.001005118119341339,0.5597220650459199,0.5764947443228802,0.5648435507167102,0.5648435507167102,0.004709970715400788,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192]",1,64,64,65,64,8192.0,0.015384615384615385,q1k_64q1s8k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018112000077962875,0.026496000587940216,0.019318400137126445,0.018432000651955605,0.0024249605112359905,0.19770253574610402,0.222811797868348,0.2054811520619412,0.2054811520619412,0.008000944254868043,0.13941746080159492,0.157124211777114,0.14490284788179197,0.14490284788179197,0.005642170080515992,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q2k_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.02595200017094612,0.03155200183391571,0.02727359998971224,0.026575999334454536,0.0016260948764843166,0.6014684881116659,0.6095472988212,0.6046757700946376,0.6046757700946376,0.0023537435563177303,0.11325152264578045,0.11477269562841545,0.11385542721485654,0.11385542721485654,0.0004431903698023628,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,16,16,17,16,4096.0,0.058823529411764705,q4k_16q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014303999952971935,0.03359999880194664,0.01814719969406724,0.015375999733805656,0.005678506740662529,0.06774400174617767,0.07891199737787247,0.07312640026211739,0.07312640026211739,0.003826583051422731,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,2,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512]",1024,512,512,512.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01788800023496151,0.028543999418616295,0.021340799890458582,0.019504000432789326,0.003718627037219618,0.09548799693584442,0.1327359974384308,0.10823359936475753,0.10823359936475753,0.013230635292043864,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,4,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512]",2048,512,512,512.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02627200074493885,0.030368000268936157,0.027184000052511693,0.02643200010061264,0.0013545679205210022,0.14364799857139587,0.1597760021686554,0.15008639842271806,0.15008639842271806,0.005367723415171222,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512]",4096,512,512,512.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04150399938225746,0.04726399853825569,0.042950399965047834,0.04224000126123428,0.001720147501765378,0.2433920055627823,0.28963199257850647,0.2549152016639709,0.2549152016639709,0.013450991019687407,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512]",8192,512,512,512.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.025887999683618546,0.03407999873161316,0.028303999826312064,0.026367999613285065,0.0028877495368841042,0.325439989566803,0.3441599905490875,0.33442879617214205,0.33442879617214205,0.005693597811441922,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,2,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048]",4096,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04137599840760231,0.05084799975156784,0.044828799366950986,0.043087998405098915,0.003560363865797613,0.6110399961471558,0.6421759724617004,0.6204223990440368,0.6204223990440368,0.011214490370794758,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,4,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048, 2048, 2048]",8192,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014816000126302242,0.02364799939095974,0.016668799985200166,0.01592000015079975,0.0025515238179941247,0.0,0.0,0.0,0.0,0.0,0.0544000007212162,0.09014400094747543,0.06511679962277411,0.06511679962277411,0.013005726711295521,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.0208320003002882,0.016604799963533878,0.01566399959847331,0.0021353594266203244,0.0,0.0,0.0,0.0,0.0,0.17871999740600586,0.1961279958486557,0.18568639904260634,0.18568639904260634,0.004566142885274747,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014431999996304512,0.02006400004029274,0.015619200002402068,0.01508800033479929,0.001572984783715635,0.0,0.0,0.0,0.0,0.0,0.2730880081653595,0.29721599817276,0.2815328001976013,0.2815328001976013,0.007016534727616923,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.02755199931561947,0.018678399827331306,0.015919999685138464,0.004323555372382858,0.0,0.0,0.0,0.0,0.0,0.3893119990825653,0.44041600823402405,0.40225600004196166,0.40225600004196166,0.013442998165884852,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014303999952971935,0.02659199945628643,0.016748800035566093,0.015343999955803156,0.0036412341868394082,0.0,0.0,0.0,0.0,0.0,0.7404800057411194,0.9689919948577881,0.8451807916164398,0.8451807916164398,0.08637455920036795,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014751999638974667,0.02160000056028366,0.016752000153064727,0.01521599991247058,0.0025167650350367246,0.0,0.0,0.0,0.0,0.0,0.06412799656391144,0.08956799656152725,0.07242240011692047,0.07242240011692047,0.009797120588901621,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.022911999374628067,0.016137600038200618,0.015343999955803156,0.002330911555470513,0.0,0.0,0.0,0.0,0.0,0.3128319978713989,0.3282879889011383,0.3203647971153259,0.3203647971153259,0.004598568238907996,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.03574400022625923,0.01919359974563122,0.01583999954164028,0.006642521913489215,0.0,0.0,0.0,0.0,0.0,0.5050879716873169,0.5311999917030334,0.511932796239853,0.511932796239853,0.007150929647317677,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.02595200017094612,0.016992000117897987,0.015456000342965126,0.003390731937723684,0.0,0.0,0.0,0.0,0.0,0.7385600209236145,0.7681919932365417,0.7483008027076722,0.7483008027076722,0.010214373356208012,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.038975998759269714,0.01952639976516366,0.015327999833971262,0.007695927285621061,0.0,0.0,0.0,0.0,0.0,1.4228800535202026,1.5237760543823242,1.4435008168220522,1.4435008168220522,0.0342955291725742,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.03907199949026108,0.01793599994853139,0.015248000156134367,0.007167103363234667,0.0,0.0,0.0,0.0,0.0,0.06947200000286102,0.08857599645853043,0.07469440028071403,0.07469440028071403,0.00583249952929747,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.02175999991595745,0.016710399929434062,0.015327999833971262,0.002705383977079183,0.0,0.0,0.0,0.0,0.0,0.388480007648468,0.4073280096054077,0.3966591984033584,0.3966591984033584,0.004487315516095964,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.0144640002399683,0.022048000246286392,0.017011200170964004,0.015392000321298838,0.002776899649845722,0.0,0.0,0.0,0.0,0.0,0.622048020362854,0.6347839832305908,0.6262047946453094,0.6262047946453094,0.004593360122958751,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014592000283300877,0.02937600016593933,0.01847040019929409,0.015344000421464443,0.005043809142122341,0.0,0.0,0.0,0.0,0.0,0.9105280041694641,0.9304640293121338,0.914108806848526,0.914108806848526,0.00563919268816644,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01462399959564209,0.039903998374938965,0.01791359977796674,0.015440000221133232,0.007367547649297217,0.0,0.0,0.0,0.0,0.0,1.764799952507019,1.7965760231018066,1.770739197731018,1.770739197731018,0.009082122770320404,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015168000012636185,0.03187200054526329,0.018553599901497363,0.016127999871969223,0.0050628774268006264,0.16963527081512533,0.1737871320906266,0.17088926838108623,0.17088926838108623,0.0011703114258121128,0.47362872483230506,0.48522089403242025,0.47712993814280913,0.47712993814280913,0.0032675581298664976,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,520,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,8,8,9,8,16384.0,0.1111111111111111,q512_8q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.03868800029158592,0.01921919994056225,0.016959999687969685,0.006691309533030663,0.1589600576212269,0.16117782913137854,0.15969057520605917,0.15969057520605917,0.0006212984448748182,0.5199519263456005,0.5272061673552852,0.5223414198520004,0.5223414198520004,0.002032242112153396,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1040,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,16,16,17,16,16384.0,0.058823529411764705,q1k_16q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018400000408291817,0.024320000782608986,0.019910399988293647,0.0191040001809597,0.001841532763984352,0.25650752966102763,0.27066608538463055,0.259736894547936,0.259736894547936,0.00462927315947332,0.5278764825612624,0.5570139063374764,0.5345223138928448,0.5345223138928448,0.009526755161797178,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,32,32,33,32,16384.0,0.030303030303030304,q2k_32q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.026048000901937485,0.03728000074625015,0.028636799938976765,0.02705600019544363,0.003313953649011259,0.9270006318443208,0.9712965120641314,0.93480767601568,0.93480767601568,0.012612551450284608,0.8181833128578277,0.8572794566782395,0.8250739157811953,0.8250739157811953,0.01113200873299586,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,16,16,17,16,32768.0,0.058823529411764705,q4k_16q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018079999834299088,0.037696000188589096,0.022092800214886667,0.019024000503122807,0.005986278786633071,0.2839857165542317,0.2876110048757805,0.2849506939696605,0.2849506939696605,0.0009995178425916847,1.0871823008331585,1.101060989333509,1.0908765231323903,1.0908765231323903,0.0038264533900426207,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,32,32,33,32,32768.0,0.030303030303030304,q2k_32q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.015072000212967396,0.025567999109625816,0.01726400014013052,0.015728000551462173,0.0031324465940990903,0.137270464802061,0.13799613818579368,0.13751499486424038,0.13751499486424038,0.00025221761530472904,2.3021855212208884,2.314355908780759,2.3062865750744197,2.3062865750744197,0.004229983070201486,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,64,64,65,64,32768.0,0.015384615384615385,q1k_64q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014560000039637089,0.022784000262618065,0.016435200069099664,0.015519999898970127,0.0023688301421469523,0.04879999905824661,0.09139200299978256,0.06054079942405224,0.06054079942405224,0.012152608702448775,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,64,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,False,0.0,64.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014816000126302242,0.02844800055027008,0.017008000146597625,0.015696000307798386,0.003941466294662679,0.047807998955249786,0.07356800138950348,0.05626560002565384,0.05626560002565384,0.00842179125412236,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,128,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,False,0.0,128.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.053408000618219376,0.019353600218892097,0.01532800029963255,0.01138302400841626,0.048448000103235245,0.0785600021481514,0.0556256003677845,0.0556256003677845,0.009348274502616908,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,256,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,False,0.0,256.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015039999969303608,0.03139200061559677,0.01823679991066456,0.016159999649971724,0.004860071238302512,0.055424001067876816,0.08505599945783615,0.0640383992344141,0.0640383992344141,0.00921448636178623,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,512,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,False,0.0,512.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014751999638974667,0.026528000831604004,0.017324799951165915,0.01536000007763505,0.0036004822686428305,0.07660800218582153,0.0942080020904541,0.08209280073642732,0.08209280073642732,0.00515895587669752,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.021247999742627144,0.016403199825435876,0.01532800029963255,0.001995897022647934,0.11395200341939926,0.15113599598407745,0.12431039959192276,0.12431039959192276,0.011164431123683732,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.017184000462293625,0.028672000393271446,0.019865600019693376,0.017823999747633934,0.0035798491315929977,0.3171840012073517,0.3341119885444641,0.3261695951223373,0.3261695951223373,0.005111046452878918,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024639999493956566,0.030400000512599945,0.02656640000641346,0.02556800004094839,0.0020189846603237303,1.0648640394210815,1.0828479528427124,1.0715327858924866,1.0715327858924866,0.005558639263049851,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014976000413298607,0.021695999428629875,0.017305599898099898,0.01593599934130907,0.0025718046517268054,0.04956800118088722,0.07932800054550171,0.06228480041027069,0.06228480041027069,0.01027160349757827,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,64,448.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,True,448.0,512.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64s512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01539199985563755,0.03580800071358681,0.019686400331556796,0.017136000096797943,0.005918387811304003,0.05158400163054466,0.10713600367307663,0.06364160068333148,0.06364160068333148,0.015832957809696766,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,128,896.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,True,896.0,1024.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015168000012636185,0.02223999984562397,0.016672000009566545,0.015887999907135963,0.0020934945946034563,0.06521599739789963,0.08902399986982346,0.07520959973335266,0.07520959973335266,0.007913840684102929,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,256,1792.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,True,1792.0,2048.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015231999568641186,0.04028800129890442,0.019971200078725816,0.01646399963647127,0.007351305886577286,0.14467200636863708,0.16844800114631653,0.15363519936800005,0.15363519936800005,0.008174435899956223,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,512,3584.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,True,3584.0,4096.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015519999898970127,0.02304000034928322,0.01775679988786578,0.016784000210464,0.0025077243712082584,0.33740800619125366,0.35343998670578003,0.3445120006799698,0.3445120006799698,0.00445648463590358,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,1024,7168.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,True,7168.0,8192.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1ks8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015647999942302704,0.02393599972128868,0.01809599995613098,0.01654400024563074,0.002998393102466254,0.0,0.0,0.0,0.0,0.0,0.0504320003092289,0.0843840017914772,0.060083200410008426,0.060083200410008426,0.00986959318572296,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01603199914097786,0.030047999694943428,0.019510399922728537,0.017680000513792038,0.004065185265963332,0.0,0.0,0.0,0.0,0.0,0.05004800111055374,0.07036799937486649,0.059315200522542,0.059315200522542,0.006537768821329647,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.02393599972128868,0.018035199772566558,0.016512000001966953,0.0030667689116777724,0.0,0.0,0.0,0.0,0.0,0.05100800096988678,0.06735999882221222,0.058387200161814694,0.058387200161814694,0.005832787739241381,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01500799972563982,0.026208000257611275,0.017500799987465142,0.01646399963647127,0.0030666103306165714,0.0,0.0,0.0,0.0,0.0,0.05023999884724617,0.07254400104284286,0.057254400476813315,0.057254400476813315,0.006890853653068151,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.027327999472618103,0.0179776000790298,0.01648000068962574,0.003783417447531392,0.0,0.0,0.0,0.0,0.0,0.04819199815392494,0.07100799679756165,0.05621119923889638,0.05621119923889638,0.007824391484124725,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015647999942302704,0.036448001861572266,0.019136000238358975,0.016704000532627106,0.006076530095624803,0.0,0.0,0.0,0.0,0.0,0.047488000243902206,0.08441600203514099,0.0588383998721838,0.0588383998721838,0.011275940726332522,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015135999768972397,0.033952001482248306,0.02119360016658902,0.018000000156462193,0.007136646614305304,0.0,0.0,0.0,0.0,0.0,0.04931199923157692,0.06992000341415405,0.05755840018391609,0.05755840018391609,0.007297587694315226,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.034272000193595886,0.0204415999352932,0.017311999574303627,0.00695404701803013,0.0,0.0,0.0,0.0,0.0,0.05215999856591225,0.0735040009021759,0.06117440015077591,0.06117440015077591,0.007381118436894922,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015359999611973763,0.03743999823927879,0.02012479966506362,0.01775999926030636,0.006181045509079057,0.0,0.0,0.0,0.0,0.0,0.06355199962854385,0.08508799970149994,0.07019200026988984,0.07019200026988984,0.0062246580442117845,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015552000142633915,0.032607998698949814,0.01959999995306134,0.017487999983131886,0.004882475068460749,0.0,0.0,0.0,0.0,0.0,0.04918399825692177,0.0865280032157898,0.05973760038614274,0.05973760038614274,0.011546847942113974,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015168000012636185,0.02675200067460537,0.017430400010198355,0.016560000367462635,0.003243135094239819,0.0,0.0,0.0,0.0,0.0,0.052671998739242554,0.08367999643087387,0.06238719932734965,0.06238719932734965,0.011207824780630104,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01500799972563982,0.03612799942493439,0.019327999837696553,0.01688000001013279,0.006058251425153085,0.0,0.0,0.0,0.0,0.0,0.06364800035953522,0.07878399640321732,0.06935679838061332,0.06935679838061332,0.004515588328677643,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015104000456631184,0.03711999952793121,0.019670399930328132,0.017152000218629837,0.006165994646522264,0.0,0.0,0.0,0.0,0.0,0.09071999788284302,0.11507199704647064,0.10252480059862136,0.10252480059862136,0.008782544051535657,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015039999969303608,0.026528000831604004,0.018396800104528665,0.01601599995046854,0.004279788048986283,0.0,0.0,0.0,0.0,0.0,0.05331199988722801,0.07977599650621414,0.06076480001211167,0.06076480001211167,0.008761579122069606,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015359999611973763,0.02643200010061264,0.01726400004699826,0.015855999663472176,0.003210008417242029,0.0,0.0,0.0,0.0,0.0,0.062431998550891876,0.08246400207281113,0.06970879957079888,0.06970879957079888,0.007016341676068233,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014911999925971031,0.02223999984562397,0.0166015999391675,0.015887999907135963,0.00204913969129354,0.0,0.0,0.0,0.0,0.0,0.10127999633550644,0.1141119971871376,0.10621120035648347,0.10621120035648347,0.004029318311754605,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015359999611973763,0.03363199904561043,0.019305599946528675,0.016671999357640743,0.0055445582307981234,0.0,0.0,0.0,0.0,0.0,0.1319040060043335,0.15839999914169312,0.14040640145540234,0.14040640145540234,0.007998041073596942,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014368000440299511,0.04447999969124794,0.018908800091594458,0.0157279996201396,0.0086814937461721,0.0,0.0,0.0,0.0,0.0,0.06428799778223038,0.08982399851083755,0.07349760085344315,0.07349760085344315,0.008605284512197258,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015104000456631184,0.028672000393271446,0.01890560006722808,0.016080000437796116,0.004797584627815021,0.0,0.0,0.0,0.0,0.0,0.11123199760913849,0.14115199446678162,0.11942399889230729,0.11942399889230729,0.008513394706791027,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.024927999824285507,0.016915200091898442,0.015216000378131866,0.003374147499442635,0.0,0.0,0.0,0.0,0.0,0.15887999534606934,0.18111999332904816,0.16934399753808976,0.16934399753808976,0.007415181260761123,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014336000196635723,0.026944000273942947,0.017737600207328796,0.015199999790638685,0.00415598714375255,0.0,0.0,0.0,0.0,0.0,0.2192319929599762,0.23472000658512115,0.22809920012950893,0.22809920012950893,0.004730841376327335,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.05052800104022026,0.020320000313222408,0.015520000364631414,0.010604522177924678,0.024270629882498958,0.03340247625954076,0.028446344104128624,0.028446344104128624,0.0032330369099793834,0.023697370291069768,0.03261352722995356,0.027774456325453972,0.027774456325453972,0.0031566742680923386,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,72,,,,,,,,False,,,64,BF16,generic,none,CUDA_EVENT,True,[64],[0],"[512, 512, 512, 512, 512, 512, 512, 512]",1,8,8,9,8,512.0,0.1111111111111111,q64_8q1s512,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.015359999611973763,0.033663999289274216,0.019987199828028678,0.018240000121295452,0.005522929139253732,0.0259194055660947,0.03719755183990719,0.029916029687899703,0.029916029687899703,0.003966666590554318,0.027104595853449518,0.03889844644729374,0.03128396954022015,0.03128396954022015,0.004148046317967881,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,136,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,8,8,9,8,1024.0,0.1111111111111111,q128_8q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014592000283300877,0.03299200162291527,0.019840000104159115,0.016207999549806118,0.006546343008726814,0.02587869595769926,0.03763167265431482,0.030174939058162802,0.030174939058162802,0.0037303581782277364,0.026473304070195713,0.03849632587654989,0.03086826083864964,0.03086826083864964,0.003816069654529222,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,144,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,16,16,17,16,1024.0,0.058823529411764705,q128_16q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.013824000023305416,0.021023999899625778,0.016304000187665223,0.015584000386297703,0.0023236165806545476,0.031810621525966996,0.04156949936878106,0.0346749350032807,0.0346749350032807,0.003130209181143985,0.035677378270900374,0.04662250161636451,0.038889864871740294,0.038889864871740294,0.0035107033960828727,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,272,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,16,16,17,16,2048.0,0.058823529411764705,q256_16q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.02175999991595745,0.01569600012153387,0.015008000191301107,0.0020882666168036863,0.038211712107062853,0.07212229256520057,0.04364509673334097,0.04364509673334097,0.009671953074025085,0.0476442859917874,0.08992570455184198,0.05441890342616107,0.05441890342616107,0.01205947791784038,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,288,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,32,32,33,32,2048.0,0.030303030303030304,q256_32q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014368000440299511,0.033824000507593155,0.017628800217062236,0.014864000026136637,0.005791495403857738,0.05214261250030033,0.06266261508706669,0.05677791295527661,0.05677791295527661,0.0030298223079651514,0.08648138506878382,0.10392938682791132,0.09416928531647478,0.09416928531647478,0.005025126612204489,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,544,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q512_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.031199999153614044,0.017900799959897996,0.015343999955803156,0.004974475661316249,0.06411958891421111,0.07405276123263956,0.06793047918211377,0.06793047918211377,0.0033918658556700006,0.14058441263169497,0.16236323591492058,0.1489399211274489,0.1489399211274489,0.00743678300375353,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,576,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,64,64,65,64,4096.0,0.015384615384615385,q512_64q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.03855999931693077,0.018495999928563833,0.015647999942302704,0.006918530056761627,0.09872985549401277,0.10683454583043753,0.10185316839884552,0.10185316839884552,0.0020802254037042074,0.2743261390166379,0.2968454508984596,0.2830044295482752,0.2830044295482752,0.005780016596065092,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192]",1,64,64,65,64,8192.0,0.015384615384615385,q1k_64q1s8k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014911999925971031,0.030400000512599945,0.01977920001372695,0.01726400014013052,0.005350883464206169,0.13918872472233365,0.16279522855335207,0.1501912864839173,0.1501912864839173,0.008992185529011513,0.11892328861766266,0.13909276049083735,0.1283239123428725,0.1283239123428725,0.007682951884956939,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q2k_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01727999933063984,0.02223999984562397,0.01819519978016615,0.017680000513792038,0.0014397298270620873,0.3728044181625443,0.38295504353701676,0.3761264483787333,0.3761264483787333,0.0033660272332490977,0.07967557017401883,0.08184495665371805,0.08038555277807365,0.08038555277807365,0.0007193856241088455,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,16,16,17,16,4096.0,0.058823529411764705,q4k_16q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01500799972563982,0.034591998904943466,0.01941439984366298,0.01756799966096878,0.005541380584373771,0.05951999872922897,0.08268799632787704,0.06715519949793816,0.06715519949793816,0.006657802116288364,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,2,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512]",1024,512,512,512.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015104000456631184,0.03283200040459633,0.019180799927562477,0.016543999314308167,0.0052187911233635975,0.07199999690055847,0.08675199747085571,0.07749439924955369,0.07749439924955369,0.004849874741156772,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,4,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512]",2048,512,512,512.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01724799908697605,0.02284800074994564,0.01928640007972717,0.018400000408291817,0.0020641390157565697,0.09676799923181534,0.12310399860143663,0.10618879944086074,0.10618879944086074,0.008982738207839057,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512]",4096,512,512,512.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024224000051617622,0.03315199911594391,0.027436799928545953,0.027328000403940678,0.002679154874546328,0.14716799557209015,0.16412800550460815,0.15470399856567385,0.15470399856567385,0.005423454433987419,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512]",8192,512,512,512.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01833599992096424,0.04806400090456009,0.026252799853682517,0.022672000341117382,0.00878438440429033,0.1844799965620041,0.2072959989309311,0.19359359890222552,0.19359359890222552,0.007663539820967659,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,2,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048]",4096,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024480000138282776,0.04979199916124344,0.029081599973142146,0.025679999962449074,0.007227422044689197,0.34147199988365173,0.37968000769615173,0.3583200007677078,0.3583200007677078,0.011804422100726231,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,4,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048, 2048, 2048]",8192,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.023744000121951103,0.017753600236028434,0.016368000768125057,0.003061040281616705,0.0,0.0,0.0,0.0,0.0,0.05686400085687637,0.1090880036354065,0.06629760004580021,0.06629760004580021,0.014803727026812366,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.021088000386953354,0.01647359998896718,0.015840000472962856,0.0017964402433206,0.0,0.0,0.0,0.0,0.0,0.08540800213813782,0.09961599856615067,0.09146559983491898,0.09146559983491898,0.00447423706371901,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014560000039637089,0.04156799986958504,0.018512000143527985,0.015584000386297703,0.007849701681877904,0.0,0.0,0.0,0.0,0.0,0.17948800325393677,0.194815993309021,0.1866239994764328,0.1866239994764328,0.004156328241374654,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.029632000252604485,0.019449600111693145,0.016736000776290894,0.005126811425136928,0.0,0.0,0.0,0.0,0.0,0.27529600262641907,0.2898879945278168,0.2825664013624191,0.2825664013624191,0.004542801609674093,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.031328000128269196,0.019759999960660933,0.016367999836802483,0.006266303740589241,0.0,0.0,0.0,0.0,0.0,0.39190399646759033,0.4079039990901947,0.3987520009279252,0.3987520009279252,0.0037367727509362725,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01539199985563755,0.040031999349594116,0.0212032001465559,0.01649599988013506,0.008068547381446682,0.0,0.0,0.0,0.0,0.0,0.06588800251483917,0.07897599786520004,0.0703904002904892,0.0703904002904892,0.004214826869769361,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.022495999932289124,0.016006399970501663,0.01515199989080429,0.002265581884342885,0.0,0.0,0.0,0.0,0.0,0.1231679990887642,0.13526399433612823,0.1288223996758461,0.1288223996758461,0.004328976434897094,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014592000283300877,0.023231999948620796,0.01705600004643202,0.016048000194132328,0.002810904312746391,0.0,0.0,0.0,0.0,0.0,0.3158079981803894,0.33129599690437317,0.32348800003528594,0.32348800003528594,0.00417599076136664,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.021888000890612602,0.01708160014823079,0.01609600055962801,0.0025289234621475062,0.0,0.0,0.0,0.0,0.0,0.5103679895401001,0.5200319886207581,0.5132320046424866,0.5132320046424866,0.0035759433157749533,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.032416000962257385,0.019353600032627583,0.016176000237464905,0.0061458798960684425,0.0,0.0,0.0,0.0,0.0,0.7400320172309875,0.7516480088233948,0.7449311971664427,0.7449311971664427,0.004573461776168607,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.0144640002399683,0.02006400004029274,0.01573119992390275,0.014992000069469213,0.001654446341262991,0.0,0.0,0.0,0.0,0.0,0.07097599655389786,0.09932799637317657,0.07828159928321837,0.07828159928321837,0.009464602895232642,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.02319999970495701,0.016883199848234654,0.015775999519973993,0.0026693906156048403,0.0,0.0,0.0,0.0,0.0,0.14176000654697418,0.1598079949617386,0.15063679963350293,0.15063679963350293,0.005741662969679433,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.03203200176358223,0.022224000189453363,0.02247999981045723,0.005996176183124981,0.0,0.0,0.0,0.0,0.0,0.39180800318717957,0.41046398878097534,0.4003200054168701,0.4003200054168701,0.005552433764307601,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015072000212967396,0.028192000463604927,0.017152000125497578,0.015536000020802021,0.003847486121602065,0.0,0.0,0.0,0.0,0.0,0.6239359974861145,0.6367359757423401,0.6285343945026398,0.6285343945026398,0.004006084286606002,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.026688000187277794,0.01718079997226596,0.015488000120967627,0.0037132536813398722,0.0,0.0,0.0,0.0,0.0,0.9141119718551636,0.9721279740333557,0.927455997467041,0.927455997467041,0.02148767779232235,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014751999638974667,0.047520000487565994,0.028710400220006704,0.0266720000654459,0.010774688401598903,0.06094816381288764,0.06816969726785131,0.0644060660218149,0.0644060660218149,0.002203106589073183,0.087051838094461,0.09736630405680231,0.09199073574791852,0.09199073574791852,0.0031466818046499644,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,520,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,8,8,9,8,16384.0,0.1111111111111111,q512_8q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014911999925971031,0.02489599958062172,0.017427200078964235,0.01595200039446354,0.003071906489592143,0.19733789497223914,0.2016295616652644,0.19860388994013833,0.19860388994013833,0.001512389485439305,0.44861409134062713,0.45837046456077934,0.45149211526120153,0.45149211526120153,0.0034381598874302305,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1040,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,16,16,17,16,16384.0,0.058823529411764705,q1k_16q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.015039999969303608,0.03046399913728237,0.020643199887126686,0.016944000497460365,0.006320148675595729,0.2157924314537054,0.22215709640166995,0.21769400765743155,0.21769400765743155,0.0019679921844645526,0.4905115822753901,0.5049789194903589,0.4948340005651485,0.4948340005651485,0.0044733865493072344,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,32,32,33,32,16384.0,0.030303030303030304,q2k_32q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.017152000218629837,0.02425600029528141,0.01865920014679432,0.018112000077962875,0.0019454553198986453,0.7430866512973927,0.7578834738720781,0.7455351005236347,0.7455351005236347,0.00416692487556653,0.7369773831645824,0.7516525540362471,0.7394057025273603,0.7394057025273603,0.004132666607961175,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,16,16,17,16,32768.0,0.058823529411764705,q4k_16q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.015263999812304974,0.02470399998128414,0.018009600043296815,0.01657600048929453,0.0031544532774534346,0.25251173919752334,0.25566890792461106,0.2536729054481981,0.2536729054481981,0.0010279562245342983,1.0425282722084215,1.0555630628624373,1.0473223013846877,1.0473223013846877,0.004244053880724073,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,32,32,33,32,32768.0,0.030303030303030304,q2k_32q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01500799972563982,0.027807999402284622,0.01912960009649396,0.01643200032413006,0.004688164695934997,0.1251379565220268,0.14341821167748953,0.12742497577885914,0.12742497577885914,0.005353812167343766,1.1355340167064276,1.3014137556763312,1.1562870179154463,1.1562870179154463,0.048581869194943325,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,64,64,65,64,32768.0,0.015384615384615385,q1k_64q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.07446400076150894,0.08246400207281113,0.07736000046133995,0.07713599875569344,0.002301031358835964,11.935359954833984,12.102368354797363,11.96896333694458,11.96896333694458,0.05162549713370252,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,8192,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,8192.0,16384.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07664000242948532,0.08118399977684021,0.07796800062060356,0.077504001557827,0.0013681081693640953,19.84774398803711,20.15795135498047,19.915702438354494,19.915702438354494,0.0912299324030902,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,8192,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,16384.0,24576.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks24k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.0753600001335144,0.08207999914884567,0.07715519964694977,0.07595199719071388,0.0024238358447475007,27.781503677368164,27.9836483001709,27.844886589050287,27.844886589050287,0.061307010154819624,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,8192,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,24576.0,32768.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04339199885725975,0.050016000866889954,0.04504639990627766,0.04391999915242195,0.002339637813151782,5.1544318199157715,5.269279956817627,5.16938238143921,5.16938238143921,0.03363105774712194,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,4096,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,8192.0,12288.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks12k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04291199892759323,0.058848001062870026,0.045657599717378615,0.04383999854326248,0.004522763233004815,9.256383895874023,9.27734375,9.261776161193849,9.261776161193849,0.0062232312957398745,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,4096,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,16384.0,20480.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks20k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.043296001851558685,0.04982399940490723,0.045123199746012685,0.04387199878692627,0.0023369398894319345,13.365216255187988,13.634464263916016,13.398719978332519,13.398719978332519,0.0787651922310212,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,4096,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,24576.0,28672.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks28k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02703999914228916,0.0297279991209507,0.027692800015211107,0.02723200060427189,0.000851127720159845,2.387968063354492,2.4014720916748047,2.3920736074447637,2.3920736074447637,0.003552645719470337,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,2048,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,8192.0,10240.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks10k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02691200003027916,0.029823999851942062,0.027689599990844728,0.027583999559283257,0.0007783369959809023,4.440767765045166,4.4521918296813965,4.444268751144409,4.444268751144409,0.004022325406840951,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,2048,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,16384.0,18432.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks18k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02703999914228916,0.03888000175356865,0.030262400023639204,0.02801600005477667,0.0037877256209144718,6.500351905822754,6.600607872009277,6.524902391433716,6.524902391433716,0.03321667087212851,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,2048,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,24576.0,26624.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks26k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07865600287914276,0.08668799698352814,0.08130879923701287,0.07993599772453308,0.0027016201268628523,35.717376708984375,35.843265533447266,35.771837615966795,35.771837615966795,0.04024815379149301,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,8192,32768.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,32768.0,40960.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04179200157523155,0.04864000156521797,0.0436256006360054,0.04267200082540512,0.002191476789190783,6.103871822357178,6.18287992477417,6.118390369415283,6.118390369415283,0.022924175047267112,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,8192,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,8192.0,16384.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04163200035691261,0.05987200140953064,0.04625920057296753,0.04403200000524521,0.005544136325859629,10.207136154174805,11.073247909545898,10.326777648925782,10.326777648925782,0.25139335734700863,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,8192,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,16384.0,24576.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks24k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04137599840760231,0.05027199909090996,0.04355199970304966,0.042399998754262924,0.0026572716942034787,14.303423881530762,14.343520164489746,14.311049747467042,14.311049747467042,0.01137511287661936,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,8192,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,24576.0,32768.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.026016000658273697,0.03718400001525879,0.03212799951434135,0.03270399942994118,0.003953184738275612,2.6534719467163086,2.6875839233398438,2.661257576942444,2.661257576942444,0.009372641904054613,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,4096,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,8192.0,12288.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks12k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.025887999683618546,0.03347200155258179,0.027168000116944313,0.02649599965661764,0.0021492000999850562,4.70630407333374,4.7400641441345215,4.714492845535279,4.714492845535279,0.009349196197962609,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,4096,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,16384.0,20480.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks20k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.025631999596953392,0.030719999223947525,0.02736639976501465,0.02711999975144863,0.0013921874495504173,6.759712219238281,6.831999778747559,6.774675178527832,6.774675178527832,0.021507023842020512,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,4096,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,24576.0,28672.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks28k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.017855999991297722,0.024831999093294144,0.01991359982639551,0.018655999563634396,0.0023971416189370203,1.3446400165557861,1.3609600067138672,1.3508928060531615,1.3508928060531615,0.006186954712541735,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,2048,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,8192.0,10240.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks10k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.0180479995906353,0.022752000018954277,0.019507200084626676,0.01896000001579523,0.0015079573553847933,2.5208001136779785,2.5887041091918945,2.534086418151855,2.534086418151855,0.01891953046799999,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,2048,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,16384.0,18432.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks18k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.018144000321626663,0.03222399950027466,0.02052800003439188,0.018864000216126442,0.004114364828004189,3.6922879219055176,3.760576009750366,3.706630396842957,3.706630396842957,0.019888657921309623,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,2048,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,24576.0,26624.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks26k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04169600084424019,0.04822399839758873,0.0439775999635458,0.04334400035440922,0.002101844179466219,18.41494369506836,18.502975463867188,18.445004844665526,18.445004844665526,0.026939913540867878,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,8192,32768.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,32768.0,40960.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.023615999147295952,0.03267199918627739,0.026281599886715412,0.0248800003901124,0.003167969318232417,3.1188158988952637,3.1837120056152344,3.133536005020142,3.133536005020142,0.018365009771617643,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,8192,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,8192.0,16384.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02380800060927868,0.03283200040459633,0.025670399703085423,0.024639999493956566,0.0026095917641781453,5.1729278564453125,5.243135929107666,5.192828798294067,5.192828798294067,0.01984737475315108,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,8192,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,16384.0,24576.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks24k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024032000452280045,0.03167999908328056,0.02665280010551214,0.02550400048494339,0.002510131946267113,7.223167896270752,7.257152080535889,7.231532812118529,7.231532812118529,0.009418898846297825,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,8192,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,24576.0,32768.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01696000061929226,0.023679999634623528,0.01866880003362894,0.017455999739468098,0.002277860345236006,1.478559970855713,1.496127963066101,1.48257919549942,1.48257919549942,0.005278358037488471,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,4096,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,8192.0,12288.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks12k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.017023999243974686,0.030688000842928886,0.02055360022932291,0.018240000121295452,0.004065815389665311,2.651711940765381,2.6691839694976807,2.657548785209656,2.657548785209656,0.005897264516465086,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,4096,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,16384.0,20480.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks20k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016767999157309532,0.024351999163627625,0.01840319987386465,0.017167999409139156,0.0025234367788448957,3.823551893234253,3.8631999492645264,3.8335999727249135,3.8335999727249135,0.011858923917429712,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,4096,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,24576.0,28672.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks28k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013952000066637993,0.03622400015592575,0.018214400112628936,0.014719999860972166,0.007277897466521675,0.7039039731025696,0.7163199782371521,0.7077983975410462,0.7077983975410462,0.004048424224033295,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,2048,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,8192.0,10240.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks10k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015104000456631184,0.03097599931061268,0.020870400313287973,0.02054399996995926,0.005422197456220526,1.2929600477218628,1.313088059425354,1.3006752014160157,1.3006752014160157,0.00790622446009414,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,2048,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,16384.0,18432.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks18k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.03232000023126602,0.019132800027728082,0.01673599984496832,0.0050857949387988245,1.8775999546051025,1.9125759601593018,1.8867840051651,1.8867840051651,0.01196043526228758,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,2048,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,24576.0,26624.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks26k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.023584000766277313,0.03126399964094162,0.02633600030094385,0.025200000032782555,0.0025710957466677864,9.283391952514648,9.48249626159668,9.331705665588379,9.331705665588379,0.06366970422148335,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,8192,32768.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,32768.0,40960.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07552000135183334,0.0851840004324913,0.07839040011167527,0.07703999802470207,0.0030805904904383677,43.62025451660156,43.89299011230469,43.68390693664551,43.68390693664551,0.08458163192147439,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,8192,40960.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,40960.0,49152.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks48k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07718399912118912,0.0854400023818016,0.07895359992980958,0.07791999727487564,0.0024425064759109735,59.421791076660156,59.73017501831055,59.5021312713623,59.5021312713623,0.10367381308510447,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,8192,57344.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,57344.0,65536.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks64k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07875200361013412,0.09055999666452408,0.08208959847688675,0.08087999746203423,0.0033614630055883482,75.2852783203125,75.55481719970703,75.38758392333985,75.38758392333985,0.08870109119325344,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,8192,73728.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,73728.0,81920.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks80k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07788799703121185,0.09888000041246414,0.0819871999323368,0.08008000254631042,0.005835618489111142,91.13442993164062,91.45164489746094,91.24225158691405,91.24225158691405,0.1009018902177725,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,8192,90112.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,90112.0,98304.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks96k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07596799731254578,0.08790399879217148,0.07967040091753005,0.07808000221848488,0.0036209252740976605,106.89055633544922,107.34063720703125,106.9560287475586,106.9560287475586,0.13032874360676439,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,8192,106496.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,106496.0,114688.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks112k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07648000121116638,0.08534400165081024,0.07928640022873878,0.07787200063467026,0.0028305104107121735,122.71724700927734,122.9840316772461,122.8334243774414,122.8334243774414,0.09625274868603513,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,8192,122880.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,122880.0,131072.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks128k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07664000242948532,0.08774399757385254,0.08081279993057251,0.07972799986600876,0.003487270970977775,130.6565399169922,131.09359741210938,130.79229431152345,130.79229431152345,0.13138911961272914,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,8192,131072.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,131072.0,139264.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks136k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01532800029963255,0.040063999593257904,0.019459199998527764,0.01601599995046854,0.007496165774486326,9.443936347961426,9.649087905883789,9.489968109130858,9.489968109130858,0.05961646816296781,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,512,130560.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,True,130560.0,131072.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512s128k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02691200003027916,0.040832001715898514,0.02942080032080412,0.027888000011444092,0.004040048279091587,16.753759384155273,16.90662384033203,16.78766403198242,16.78766403198242,0.04236470233451937,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,2048,65536.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,65536.0,67584.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks66k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04416000097990036,0.04867200180888176,0.04565120078623295,0.04468800127506256,0.0018079971851529544,50.29715347290039,50.50300979614258,50.35054740905761,50.35054740905761,0.0724480602714254,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,4096,98304.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,98304.0,102400.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks100k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.060447998344898224,0.06684800237417221,0.06276160031557083,0.061824001371860504,0.0024045636532566625,96.07142639160156,96.53209686279297,96.1898666381836,96.1898666381836,0.1365794879996899,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,147456,1,6144,131072.0,True,FLASH_ATTN,False,vllm020_batch_spec,[6144],6144,6144,6144,6144.0,True,0.0,0.0,0.0,True,131072.0,137216.0,6144,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q6ks134k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04217600077390671,0.05100800096988678,0.04406719990074635,0.042847998440265656,0.002656672983576728,22.5166072845459,22.986656188964844,22.63068161010742,22.63068161010742,0.13840494695459715,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,8192,40960.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,40960.0,49152.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks48k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04182400181889534,0.047200001776218414,0.043644800409674646,0.04262400045990944,0.001927741871473188,30.725727081298828,30.787456512451172,30.74285774230957,30.74285774230957,0.019110324860515015,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,8192,57344.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,57344.0,65536.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks64k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04185599833726883,0.05049600079655647,0.043852799385786054,0.04289599880576134,0.0024573088797598033,38.930206298828125,38.98448181152344,38.94538269042969,38.94538269042969,0.017734743323340796,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,8192,73728.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,73728.0,81920.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks80k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04227200150489807,0.046911999583244324,0.04412479996681214,0.04327999986708164,0.001811858548765738,47.13151931762695,47.245887756347656,47.148198699951166,47.148198699951166,0.033297799765613076,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,8192,90112.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,90112.0,98304.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks96k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04182400181889534,0.06195199862122536,0.04529919996857643,0.043136000633239746,0.005751677082163394,55.338497161865234,55.64672088623047,55.42207336425782,55.42207336425782,0.11010194068839292,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,8192,106496.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,106496.0,114688.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks112k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04182400181889534,0.054687999188899994,0.04418559968471527,0.042767999693751335,0.0036820249064621804,63.549087524414055,63.80697631835938,63.62084197998047,63.62084197998047,0.08872184973119365,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,8192,122880.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,122880.0,131072.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks128k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.0461760014295578,0.06566400080919266,0.05008000023663044,0.04843199998140335,0.0055350567765421,67.64147186279297,67.91651153564453,67.70619888305666,67.70619888305666,0.0993520482760623,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,8192,131072.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,131072.0,139264.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks136k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.023711999878287315,0.03859199956059456,0.02689919974654913,0.0244159996509552,0.004626699769228834,4.775519847869873,4.829855918884277,4.790303993225097,4.790303993225097,0.01597276061319242,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,512,130560.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,True,130560.0,131072.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512s128k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024224000051617622,0.0453759990632534,0.027676799893379213,0.025200000032782555,0.006154880946840619,9.579392433166504,9.63871955871582,9.59802885055542,9.59802885055542,0.016811667352984866,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,2048,65536.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,65536.0,67584.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks66k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02630399912595749,0.03593600168824196,0.02780479993671179,0.027040000073611736,0.0027569987978884785,25.21686363220215,25.383039474487305,25.250255966186526,25.250255966186526,0.04761963064809279,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,4096,98304.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,98304.0,102400.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks100k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.034015998244285583,0.041728001087903976,0.03665280006825924,0.0352960005402565,0.002710617869728372,48.101280212402344,48.49884796142578,48.17219200134278,48.17219200134278,0.11195495368044632,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,147456,1,6144,131072.0,True,FLASH_ATTN,False,vllm020_batch_spec,[6144],6144,6144,6144,6144.0,True,0.0,0.0,0.0,True,131072.0,137216.0,6144,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q6ks134k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024191999807953835,0.039583999663591385,0.027286400087177753,0.025200000032782555,0.004620118437533604,11.32140827178955,11.496224403381348,11.354345512390138,11.354345512390138,0.050117766215439966,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,8192,40960.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,40960.0,49152.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks48k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024032000452280045,0.03232000023126602,0.026406400091946124,0.02518399991095066,0.002792696117638337,15.42249584197998,15.639776229858397,15.451993656158448,15.451993656158448,0.06304729308178388,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,8192,57344.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,57344.0,65536.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks64k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024224000051617622,0.03488000109791756,0.026147199980914592,0.02459200005978346,0.003303059079404486,19.5251522064209,19.70569610595703,19.54985942840576,19.54985942840576,0.052248914965338324,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,8192,73728.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,73728.0,81920.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks80k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024159999564290047,0.03046399913728237,0.0255103999748826,0.024656000547111034,0.00195717586616911,23.630016326904297,23.927040100097656,23.672822570800783,23.672822570800783,0.08715594728873713,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,8192,90112.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,90112.0,98304.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks96k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02380800060927868,0.03574400022625923,0.02678720001131296,0.024720000103116035,0.003678231234048846,27.736671447753906,27.79840087890625,27.747158622741697,27.747158622741697,0.018952179343788657,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,8192,106496.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,106496.0,114688.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks112k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024224000051617622,0.042080000042915344,0.02844479978084564,0.02527999971061945,0.006562862750709003,31.83427238464356,32.08835220336914,31.86994876861572,31.86994876861572,0.07444245241186317,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,8192,122880.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,122880.0,131072.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks128k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02377600036561489,0.03139200061559677,0.026396799832582474,0.025071999989449978,0.0025962810796740263,33.88313674926758,33.96985626220703,33.89708137512208,33.89708137512208,0.02518493598134421,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,8192,131072.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,131072.0,139264.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks136k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014944000169634819,0.022624000906944275,0.017174400109797715,0.015536000020802021,0.0028331860349340154,3.179744005203247,3.192960023880005,3.182867193222046,3.182867193222046,0.0035239767128429386,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,512,130560.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,True,130560.0,131072.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512s128k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.02985600009560585,0.017318400088697672,0.015568000264465809,0.004479309643844885,4.811935901641846,4.826848030090332,4.8173023700714115,4.8173023700714115,0.00594674080057053,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,2048,65536.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,65536.0,67584.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks66k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.017184000462293625,0.03916800022125244,0.021356799826025962,0.01935999933630228,0.0062801287963799276,14.374591827392578,14.635007858276367,14.41227512359619,14.41227512359619,0.07569270440104603,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,4096,98304.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,98304.0,102400.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks100k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.020479999482631683,0.033215999603271484,0.023340800032019614,0.02092800009995699,0.004304974035052475,24.088096618652344,24.18492889404297,24.11243553161621,24.11243553161621,0.02797845886790172,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,147456,1,6144,131072.0,True,FLASH_ATTN,False,vllm020_batch_spec,[6144],6144,6144,6144,6144.0,True,0.0,0.0,0.0,True,131072.0,137216.0,6144,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q6ks134k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median

runs/frontier-code-trace-v0/profiles/profile-v6-code-longctx/linear_op.csv

@@ -0,0 +1,37 @@
+time_stats.emb.min,time_stats.emb.max,time_stats.emb.mean,time_stats.emb.median,time_stats.emb.std,time_stats.input_layernorm.min,time_stats.input_layernorm.max,time_stats.input_layernorm.mean,time_stats.input_layernorm.median,time_stats.input_layernorm.std,time_stats.attn_pre_proj.min,time_stats.attn_pre_proj.max,time_stats.attn_pre_proj.mean,time_stats.attn_pre_proj.median,time_stats.attn_pre_proj.std,time_stats.attn_rope.min,time_stats.attn_rope.max,time_stats.attn_rope.mean,time_stats.attn_rope.median,time_stats.attn_rope.std,time_stats.attn_post_proj.min,time_stats.attn_post_proj.max,time_stats.attn_post_proj.mean,time_stats.attn_post_proj.median,time_stats.attn_post_proj.std,time_stats.post_attention_layernorm.min,time_stats.post_attention_layernorm.max,time_stats.post_attention_layernorm.mean,time_stats.post_attention_layernorm.median,time_stats.post_attention_layernorm.std,n_head,n_kv_head,n_embd,n_expanded_embd,vocab_size,use_gated_mlp,use_qk_norm,attn_output_gate,num_tokens,num_tensor_parallel_workers,padded_n_embd,padded_n_expanded_embd,model_arch,is_step2_mini,share_expert_dim,share_q_dim,measurement_type,profiling_precision,quant_signature
+0.029184000566601753,0.06780800223350525,0.03157280012965202,0.030736000277101994,0.005874173435341216,0.033215999603271484,0.04825599864125252,0.03443359974771738,0.0337119996547699,0.0031825678429048183,1.438431978225708,1.505568027496338,1.446228802204132,1.4429279565811157,0.014128607642643025,0.538752019405365,0.5440319776535034,0.5416463971138,0.5420799851417542,0.0016099306164432847,1.0073280334472656,1.0163840055465698,1.0098415970802308,1.0081279873847961,0.0032980457197329728,0.04022400081157684,0.041728001087903976,0.04091359991580248,0.04081599973142147,0.0004728505416767937,32,4,2048,768,151936,True,True,False,8192,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.01360000018030405,0.06784000247716904,0.017755200061947106,0.0179840000346303,0.00837681421401443,0.01836800016462803,0.03651199862360954,0.019606399815529585,0.018719999119639397,0.0038821958848767424,0.7512000203132629,0.8208960294723511,0.7566704005002975,0.75382399559021,0.014793092586577971,0.28995200991630554,0.29337599873542786,0.29135999977588656,0.29150401055812836,0.000998381071258815,0.5149760246276855,0.5169600248336792,0.5160208016633987,0.5158880054950714,0.0005038652783291977,0.02051199972629547,0.021503999829292297,0.021067200042307378,0.021104000508785248,0.00023542855306902367,32,4,2048,768,151936,True,True,False,4096,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.0080960001796484,0.04281599819660187,0.011092799971811474,0.011039999779313803,0.005489135752949302,0.012223999947309494,0.026335999369621277,0.013187199970707298,0.01247999956831336,0.0030236510562153375,0.3928639888763428,0.45372799038887024,0.3976895987987518,0.39528000354766846,0.012905176716136006,0.1547199934720993,0.15884800255298615,0.15712319910526276,0.1573439985513687,0.001165121945135037,0.26633599400520325,0.268095999956131,0.26719200164079665,0.2671840041875839,0.0004242740199415328,0.013024000450968742,0.013887999579310417,0.013489600038155913,0.013520000036805868,0.0002382817554057738,32,4,2048,768,151936,True,True,False,2048,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.00825599953532219,0.04755200073122978,0.02398160002194345,0.01961600035429001,0.00874683840888523,0.018880000337958336,0.03868800029158592,0.021590400114655496,0.0208320003002882,0.004057015751746236,0.24316799640655518,0.2710399925708771,0.2521967992186546,0.25065599381923676,0.007998418528894075,0.09644799679517746,0.19120000302791595,0.10407840013504029,0.09963199868798256,0.020043722414992166,0.13600000739097595,0.18892799317836761,0.15760480016469955,0.15760000050067902,0.0112223677907762,0.009664000011980534,0.010015999898314476,0.009836799977347255,0.009824000298976898,9.016971952948177e-05,32,4,2048,768,151936,True,True,False,1024,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.017376000061631203,0.044704001396894455,0.026318399980664254,0.027312000282108784,0.007565344034680866,0.01836800016462803,0.03014400042593479,0.021276800055056812,0.020655999891459942,0.002632977855097951,0.1438719928264618,0.17132799327373505,0.152497598528862,0.15012799948453903,0.007947150319625347,0.1430719941854477,0.19305600225925446,0.16630879789590836,0.1685439944267273,0.014628024163894684,0.08899199962615967,0.10467199981212616,0.0943599995225668,0.09374399855732918,0.003472669494074113,0.007615999784320593,0.007935999892652035,0.007769599952735007,0.0077760000713169575,7.680004540222077e-05,32,4,2048,768,151936,True,True,False,512,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.016992000862956047,0.0544000007212162,0.02573199989274144,0.026016000658273697,0.00803323401720434,0.018432000651955605,0.02348800003528595,0.020648000109940768,0.02062400057911873,0.0013939985047930988,0.10220800340175629,0.1361600011587143,0.11850560046732425,0.11684799939393997,0.010637754898360304,0.16710400581359863,0.21110400557518005,0.19078560024499894,0.19409599900245667,0.013714352646558832,0.06265600025653839,0.0740479975938797,0.06842879951000214,0.0690080001950264,0.0031292240446560557,0.00979200005531311,0.033440001308918,0.01864320016466081,0.017280000261962414,0.006957998188876383,32,4,2048,768,151936,True,True,False,256,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.017152000218629837,0.04396799951791763,0.025907999789342284,0.02598400041460991,0.007714554737915267,0.018400000408291817,0.03667199984192848,0.024132800102233887,0.02112000063061714,0.006120348733354326,0.10678400099277496,0.1363839954137802,0.1193264003843069,0.11583999916911125,0.009838647443214228,0.17017599940299988,0.22748799622058868,0.18853759989142418,0.18433599919080734,0.015728078443174653,0.04569600149989128,0.06652799993753433,0.05192639995366335,0.05151999928057194,0.004516605694976449,0.021856000646948814,0.026335999369621277,0.02384479995816946,0.023599999956786633,0.0011301153013314744,32,4,2048,768,151936,True,True,False,128,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.017343999817967415,1.0683200359344482,0.05748240072280168,0.029504000209271908,0.16366824814254827,0.018688000738620758,0.3317759931087494,0.03685439983382821,0.021151999942958355,0.06767422466731164,0.10255999863147736,0.9434880018234253,0.16412640027701855,0.11956800147891045,0.17964606281728834,0.1714559942483902,2.1306240558624268,0.3011296011507511,0.1926399990916252,0.42310127734378766,0.03574400022625923,0.6859520077705383,0.08389280084520578,0.04279999993741512,0.14321647071615612,0.020479999482631683,0.1831360012292862,0.033024000097066165,0.023856000043451786,0.03470987082429836,32,4,2048,768,151936,True,True,False,64,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.016095999628305435,0.05142400041222572,0.026363200135529043,0.02676799986511469,0.008637725852473854,0.01849599927663803,0.03577600046992302,0.022193600237369538,0.020848000422120094,0.004615266181181815,0.10540799796581268,0.15014399588108063,0.12211520001292228,0.11896000057458878,0.012772013396624768,0.17315199971199036,0.21478399634361267,0.1881632000207901,0.1873439997434616,0.011761164657572015,0.03481600061058998,0.058079998940229416,0.042200000025331974,0.03969600051641464,0.006461281521769989,0.02143999934196472,0.038816001266241074,0.02466559996828437,0.023856000043451786,0.0035418033748569927,32,4,2048,768,151936,True,True,False,32,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.015904000028967857,0.03753599897027016,0.023127999808639287,0.02527999971061945,0.0054230027890305385,0.018400000408291817,0.03001599945127964,0.020488000102341176,0.020096000283956528,0.002585688394137697,0.10355199873447418,0.1438400000333786,0.11536479964852334,0.11124800145626068,0.011136028294919255,0.16502399742603302,0.2072959989309311,0.18646399974822997,0.19075199961662292,0.013296437761247597,0.03142400085926056,0.05215999856591225,0.03888959977775812,0.03750399872660637,0.0057399302067536314,0.020128000527620316,0.04064000025391579,0.023937600292265417,0.023648000322282314,0.004144197400898248,32,4,2048,768,151936,True,True,False,16,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.015200000256299973,0.037151999771595,0.02144480012357235,0.02195199951529503,0.005576364091933697,0.017952000722289085,0.0226879995316267,0.019934400077909233,0.019952000118792057,0.0011812499470458758,0.10063999891281128,0.13468800485134125,0.11595199964940547,0.1207519993185997,0.011974301621092394,0.16332800686359406,0.20748800039291382,0.18162400051951408,0.1767839938402176,0.01474120743720334,0.03222399950027466,0.043455999344587326,0.03829439990222454,0.03859200142323971,0.0031378131240041122,0.020959999412298203,0.03587200120091438,0.023795200139284135,0.023423999547958374,0.0031339489695198443,32,4,2048,768,151936,True,True,False,8,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.014944000169634819,0.04022400081157684,0.02162720002233982,0.022463999688625336,0.006017219317026815,0.018464000895619392,0.026528000831604004,0.021264000236988066,0.020896000787615776,0.001934095017586082,0.10156799852848053,0.1703999936580658,0.12565439902245998,0.1244799979031086,0.01641776722785409,0.1653759926557541,0.23865599930286407,0.1969360001385212,0.19223999977111816,0.02058903050274278,0.03254399821162224,0.06752000004053116,0.0443536002188921,0.041519999504089355,0.00984829071098989,0.020096000283956528,0.040031999349594116,0.026934400014579297,0.02478400059044361,0.00562071702648593,32,4,2048,768,151936,True,True,False,1,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.7531200051307678,0.8461440205574036,0.7618160009384155,0.7576479911804199,0.019464233617982506,0.2922559976577759,0.2985599935054779,0.2953856036067009,0.29576000571250916,0.001575901077689139,0.510047972202301,0.5140479803085327,0.5121696025133133,0.5123839974403381,0.0012263137313476844,,,,,,32,4,2048,768,151936,True,True,False,8192,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.39529600739479065,0.47279998660087585,0.40216960161924364,0.39825600385665894,0.0162749796240819,0.15625600516796112,0.16211199760437012,0.15959519892930984,0.15988799929618835,0.0011493418942396922,0.2635200023651123,0.2642880082130432,0.2639120012521744,0.26392000913619995,0.0001903593401384135,,,,,,32,4,2048,768,151936,True,True,False,4096,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.2531839907169342,0.32419198751449585,0.26924319565296173,0.26049599051475525,0.01657194262368116,0.10051199793815613,0.2375359982252121,0.12411200068891048,0.10311999917030334,0.039466165428540506,0.15881599485874176,0.19289599359035492,0.16896959990262986,0.1640480011701584,0.009162416086418127,,,,,,32,4,2048,768,151936,True,True,False,2048,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.15302400290966034,0.1780800074338913,0.16284480094909667,0.16113600134849548,0.00740355661356144,0.14521600306034088,0.20233599841594696,0.17807039842009545,0.1796799972653389,0.013908448560094403,0.0907519981265068,0.09750399738550186,0.09460479989647866,0.09478399902582169,0.0017965487667361475,,,,,,32,4,2048,768,151936,True,True,False,1024,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11740799993276596,0.17958399653434753,0.13262080028653145,0.12878400087356567,0.014146060532423056,0.17468799650669098,0.21161599457263947,0.1910431995987892,0.18966399878263474,0.01270903647700971,0.05926400050520897,0.07577600330114365,0.06530559975653887,0.06404799968004227,0.00479458462514625,,,,,,32,4,2048,768,151936,True,True,False,512,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11260800063610077,0.17209599912166595,0.13610880002379416,0.13809599727392197,0.017675922458993677,0.18729600310325623,0.26822400093078613,0.20815680101513861,0.2078079953789711,0.01700718593658771,0.04499199986457825,0.06537599861621857,0.0507551996037364,0.04787199944257736,0.005727123232692158,,,,,,32,4,2048,768,151936,True,True,False,256,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11257600039243698,0.23452800512313843,0.13866880126297473,0.13964799791574478,0.026590047697062115,0.1828799992799759,0.24751999974250793,0.20735519900918006,0.20670399814844131,0.01500438131586336,0.03654399886727333,0.05593600124120712,0.0408239996060729,0.03892800025641918,0.004864409450710354,,,,,,32,4,2048,768,151936,True,True,False,128,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11734399944543839,0.18726399540901184,0.13890240006148816,0.13308800011873245,0.019086167340006257,0.16991999745368958,0.2443840056657791,0.19185120090842248,0.19257599860429764,0.018017536159219673,0.03033600002527237,0.04956800118088722,0.038387199863791466,0.03750400058925152,0.00500897018702887,,,,,,32,4,2048,768,151936,True,True,False,64,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11420799791812897,0.18115200102329254,0.13938880078494548,0.1393439993262291,0.018141313962922536,0.1693439930677414,0.221343994140625,0.19187839925289155,0.19438399374485016,0.01430752121272052,0.03017600066959858,0.06019200012087822,0.03866560012102127,0.0364960003644228,0.0074199790031205266,,,,,,32,4,2048,768,151936,True,True,False,32,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11276800185441971,0.16223999857902527,0.1331360016018152,0.13305599987506866,0.01432493740801146,0.17187200486660004,0.23625600337982178,0.19287680014967917,0.1913280040025711,0.01754628831589704,0.029823999851942062,0.07574400305747986,0.03900959976017475,0.036927999928593636,0.009277465083962879,,,,,,32,4,2048,768,151936,True,True,False,16,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.1130559965968132,0.19814400374889374,0.13379519879817964,0.12531199678778648,0.020074427302248836,0.16841599345207214,0.21139200031757355,0.1893615983426571,0.19092799723148346,0.012418720676762711,0.029503999277949333,0.07558400183916092,0.040144000016152856,0.03728000074625015,0.010223239196498205,,,,,,32,4,2048,768,151936,True,True,False,8,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.115167997777462,0.2640640139579773,0.13805920109152794,0.13118399679660797,0.03132849683515479,0.17103999853134155,0.2977280020713806,0.19899839907884598,0.1976960003376007,0.028928046763067948,0.0297279991209507,0.05990400165319443,0.03834720011800528,0.0363520011305809,0.0070885330713495905,,,,,,32,4,2048,768,151936,True,True,False,1,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.3940800130367279,0.46483200788497925,0.39923040121793746,0.3957759886980057,0.015108903906233569,0.16022400557994843,0.1634880006313324,0.16203359961509706,0.16228799521923065,0.0010220720912414007,0.26073598861694336,0.2627840042114258,0.26138080209493636,0.2613760083913803,0.00041640363494172775,,,,,,32,4,2048,768,151936,True,True,False,8192,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.2536959946155548,0.27161601185798645,0.25960480123758317,0.2577280104160309,0.005142017404245015,0.09849599748849869,0.10281600058078766,0.10057279989123344,0.10063999891281128,0.0008836232237268523,0.13913600146770477,0.17606399953365326,0.15875840038061143,0.15988799929618835,0.010058952112389172,,,,,,32,4,2048,768,151936,True,True,False,4096,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.15142400562763214,0.1780479997396469,0.16139679849147798,0.1602879986166954,0.0070656055731385115,0.14601600170135498,0.23343999683856964,0.17913119941949845,0.179967999458313,0.023065274309176566,0.09388799965381622,0.12108799815177917,0.10317599996924401,0.10073599964380264,0.007364345618470178,,,,,,32,4,2048,768,151936,True,True,False,2048,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.12198399752378464,0.18406400084495544,0.13770400024950505,0.13232000172138214,0.01623164885687898,0.1773120015859604,0.20688000321388245,0.1870912007987499,0.18535999953746796,0.00825628058448234,0.05766399949789047,0.06739199906587601,0.06187200043350458,0.061216000467538834,0.003004312467037863,,,,,,32,4,2048,768,151936,True,True,False,1024,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11737599968910217,0.18614399433135986,0.13109439946711063,0.12771200388669968,0.01372168725934724,0.17587199807167053,0.2699519991874695,0.19926720038056372,0.19075199961662292,0.02327478982490538,0.041728001087903976,0.06224000081419945,0.05000480003654957,0.049375999718904495,0.006185850944273961,,,,,,32,4,2048,768,151936,True,True,False,512,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.1159679964184761,0.1610880047082901,0.12988320142030715,0.12494400516152382,0.011865375783184065,0.1767680048942566,0.27379199862480164,0.21035519987344742,0.21275199949741364,0.023206964017550125,0.037087999284267426,0.062144000083208084,0.04439679980278015,0.04283200018107891,0.006155226392511492,,,,,,32,4,2048,768,151936,True,True,False,256,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.12015999853610992,0.158720001578331,0.13573280088603495,0.1343199983239174,0.010666830836014414,0.1737920045852661,0.22127999365329742,0.20261440128087999,0.20321600139141083,0.011153965849067301,0.03587200120091438,0.04944000020623207,0.038265600241720675,0.037328001111745834,0.0030431580113575636,,,,,,32,4,2048,768,151936,True,True,False,128,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.1212799996137619,0.15139199793338776,0.13183839991688728,0.13014400005340576,0.00851207547022104,0.17257599532604218,0.2250880002975464,0.1872655987739563,0.18193599581718445,0.013638284975248818,0.02969600073993206,0.0525440014898777,0.03840640028938651,0.03444799967110157,0.007598511754254174,,,,,,32,4,2048,768,151936,True,True,False,64,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11270400136709213,0.1624639928340912,0.13508000001311302,0.13180799782276154,0.014243564451606775,0.17052799463272095,0.23715199530124664,0.1953311987221241,0.1966560035943985,0.01500670718978398,0.030239999294281006,0.05270399898290634,0.03807039987295866,0.03742399998009205,0.004906165602021684,,,,,,32,4,2048,768,151936,True,True,False,32,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.12319999933242798,0.17919999361038208,0.14059039913117885,0.14156799763441086,0.016217662982107223,0.1701119989156723,0.21987199783325195,0.18903039917349815,0.1870879977941513,0.014117854507841239,0.030368000268936157,0.06406400352716446,0.04045119984075427,0.03710399940609932,0.00878942205983628,,,,,,32,4,2048,768,151936,True,True,False,16,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11296000331640244,0.16761599481105804,0.13843199908733367,0.13814399391412735,0.014925286935582404,0.1711679995059967,0.21561600267887115,0.1906527981162071,0.1876479983329773,0.011803992622137974,0.0306560005992651,0.09216000139713287,0.041129599791020155,0.036847999319434166,0.013739721468154687,,,,,,32,4,2048,768,151936,True,True,False,8,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11929599940776825,0.2008959949016571,0.14535359852015972,0.13971199840307236,0.024769473061576282,0.16803200542926788,0.25123199820518494,0.19573760256171227,0.19366399943828583,0.023196011081705884,0.03049599938094616,0.08089599758386612,0.04739360017701984,0.0453919991850853,0.01286389846889463,,,,,,32,4,2048,768,151936,True,True,False,1,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none

runs/frontier-code-trace-v0/profiles/profile-v6-code-longctx/manifest.json

@@ -0,0 +1,21 @@
+{
+  "schema": "frontier-profile-v6-code-longctx-v1",
+  "base": "runs/frontier-prefill-kvgrowth-fix-v0/profiles/profile-v5-kvgrowth/attention.csv",
+  "base_sha256": "ff32c38975e68565c85770d85b92bc310ab1d8e34fd02327bef7fb305a9ffae3",
+  "raw_inputs": {
+    "runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/raw/flashattn-code-longctx-tp1.json": "3937c623adcea3d379b82bb7ab63d3b293f917fa59fa6fd6a147581d6a252cb6",
+    "runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/raw/flashattn-code-longctx-tp2.json": "e48aa6e6d039bdb68759701c9976f78ad818d6778b6c40b14823a0a4e852c52d",
+    "runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/raw/flashattn-code-longctx-tp4.json": "759d33decc09d229d07818df8161ed41180cab575304f335ad4d5e7a917cbc93"
+  },
+  "appended_rows": 33,
+  "max_model_len": 147456,
+  "anchor_checks": [
+    "anchor q1ks8k/TP1: v4=1.1445ms new=1.1426ms rel_diff=0.2%",
+    "anchor q512s4k/TP1: v4=0.3391ms new=0.3367ms rel_diff=0.7%",
+    "anchor q1ks8k/TP2: v4=0.6126ms new=0.6065ms rel_diff=1.0%",
+    "anchor q512s4k/TP2: v4=0.2086ms new=0.2079ms rel_diff=0.3%",
+    "anchor q1ks8k/TP4: v4=0.3445ms new=0.3470ms rel_diff=0.7%",
+    "anchor q512s4k/TP4: v4=0.1536ms new=0.1510ms rel_diff=1.7%"
+  ],
+  "output_sha256": "fbcf7e1f95789a6f6d771e24d1fc60958b7daf04eb0db260d27869a19d71d550"
+}

runs/frontier-code-trace-v0/profiles/profile-v6-code-longctx/moe.csv

@@ -0,0 +1,73 @@
+time_stats.moe_gating_linear.min,time_stats.moe_gating_linear.max,time_stats.moe_gating_linear.mean,time_stats.moe_gating_linear.median,time_stats.moe_gating_linear.std,time_stats.moe_gating_routing_topk.min,time_stats.moe_gating_routing_topk.max,time_stats.moe_gating_routing_topk.mean,time_stats.moe_gating_routing_topk.median,time_stats.moe_gating_routing_topk.std,time_stats.moe_shuffling.min,time_stats.moe_shuffling.max,time_stats.moe_shuffling.mean,time_stats.moe_shuffling.median,time_stats.moe_shuffling.std,time_stats.moe_grouped_gemm.min,time_stats.moe_grouped_gemm.max,time_stats.moe_grouped_gemm.mean,time_stats.moe_grouped_gemm.median,time_stats.moe_grouped_gemm.std,num_tokens,num_experts,num_experts_per_device,expert_parallel_size,routing_runtime_path,routing_assignment_policy,routing_weight_policy,routing_uses_router_logits,gating_runtime_context,gating_runtime_context_impl,router_topk,hidden_dim,expert_hidden_dim,use_gated,num_tensor_parallel_workers,total_routed_tokens,model_expansion_ratio,tokens_per_expert_avg,tokens_to_experts_ratio,expert_utilization,min_load_ratio,load_imbalance_cv,max_load_ratio,load_entropy,load_gini_coefficient,load_distribution,seed,moe_grouped_gemm_backend,measurement_type,profiling_precision,model_arch,quant_signature,router_median_nonadditivity_ratio,projection_policy
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.33926400542259216,0.405023992061615,0.36780479848384856,0.36507199704647064,0.01690507644474779,1,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,1.1943039894104004,1.286784052848816,1.228384006023407,1.2273280024528503,0.02832547242381263,8,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,64,0.375,0.5,0.5,0.3984375,0.0,1.346291201783626,4.0,5.59375,0.661865234375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,1.679744005203247,1.766144037246704,1.7095808148384095,1.7015680074691772,0.02438921262998535,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,128,0.375,1.0,1.0,0.625,0.0,1.015504800579495,5.0,6.15516433212955,0.529052734375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,2.1576640605926514,2.2921600341796875,2.2097824096679686,2.188944101333618,0.045572321842012986,32,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,256,0.375,2.0,2.0,0.875,0.0,0.6343057228182637,2.5,6.64370748444639,0.35369873046875,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,2.422368049621582,2.5130879878997803,2.4516672134399413,2.434159994125366,0.03278900287381846,64,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,512,0.375,4.0,4.0,0.984375,0.0,0.4921254921257382,2.25,6.817190042344769,0.272369384765625,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,2.2217600345611572,2.289599895477295,2.2571327924728393,2.263375997543335,0.021660416089449488,128,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,1024,0.375,8.0,8.0,1.0,0.125,0.3486861500690843,1.875,6.908192310183997,0.197662353515625,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,2.18668794631958,2.318079948425293,2.2218016147613526,2.211087942123413,0.035380897213135316,256,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,2048,0.375,16.0,16.0,1.0,0.4375,0.2525504668006971,1.875,6.953347743053017,0.1410369873046875,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,2.2291839122772217,2.3929600715637207,2.2908096313476562,2.2804640531539917,0.04479348924786221,512,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,4096,0.375,32.0,32.0,1.0,0.65625,0.15765965680164504,1.5625,6.98229848728205,0.08779525756835938,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,2.0678720474243164,2.1297600269317627,2.0837119817733765,2.0779199600219727,0.017880044357986735,1024,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,8192,0.375,64.0,64.0,1.0,0.625,0.12169081635504074,1.3125,6.9892029662356325,0.06879425048828125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,2.916032075881958,3.0819520950317383,2.9805248022079467,2.9656319618225098,0.05482195799019572,2048,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,16384,0.375,128.0,128.0,1.0,0.796875,0.07935434147688751,1.1796875,6.9954297964750305,0.044734954833984375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,4.421599864959717,4.535359859466553,4.486294317245483,4.497056007385254,0.036990243787549344,4096,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,32768,0.375,256.0,256.0,1.0,0.8203125,0.060849326483103046,1.17578125,6.9973188375859685,0.033740997314453125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,7.302591800689697,7.402751922607422,7.354758310317993,7.3464319705963135,0.032142662400335566,8192,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,65536,0.375,512.0,512.0,1.0,0.890625,0.0412323087266341,1.08984375,6.998772433185578,0.02334284782409668,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.35280001163482666,0.39692801237106323,0.37662720382213594,0.37196800112724304,0.013401318050665304,1,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,0.4692479968070984,0.5523840188980103,0.5134752035140991,0.5100640058517456,0.02291433464135784,8,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,64,0.375,0.5,0.5,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,0.34652799367904663,0.4119040071964264,0.3789471983909607,0.38550400733947754,0.02073945105803335,16,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,128,0.375,1.0,1.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,0.31462401151657104,0.7456960082054138,0.38617280423641204,0.34545600414276123,0.12230201266253077,32,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,256,0.375,2.0,2.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,0.32521599531173706,0.419871985912323,0.36325119733810424,0.34968000650405884,0.03161798848223672,64,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,512,0.375,4.0,4.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,0.289792001247406,0.4663360118865967,0.4091839998960495,0.41655999422073364,0.0446001986506615,128,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,1024,0.375,8.0,8.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,0.3761279881000519,0.4416320025920868,0.40686399936676027,0.40540799498558044,0.02257778769899645,256,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,2048,0.375,16.0,16.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,0.7172480225563049,0.8663039803504944,0.7723807990550995,0.7591840028762817,0.04164772379451242,512,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,4096,0.375,32.0,32.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,1.0195519924163818,1.2216639518737793,1.1253888130187988,1.1453600525856018,0.06548005322243594,1024,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,8192,0.375,64.0,64.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,1.7490559816360474,1.9644800424575806,1.8529024004936219,1.814303994178772,0.08042565617327288,2048,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,16384,0.375,128.0,128.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,3.2479360103607178,3.385279893875122,3.296070408821106,3.2800960540771484,0.04525529026442046,4096,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,32768,0.375,256.0,256.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,6.344799995422363,6.517856121063232,6.464438438415527,6.478623867034912,0.05116674443145098,8192,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,65536,0.375,512.0,512.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.2648000121116638,0.325439989566803,0.28852800130844114,0.28390398621559143,0.01933778377635077,1,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,0.7347840070724487,0.862496018409729,0.7769344031810761,0.769216001033783,0.03290485328796285,8,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,64,0.375,0.5,0.5,0.421875,0.0,1.346291201783626,6.0,5.652114648336087,0.636962890625,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,0.9198399782180786,0.9646080136299133,0.9412063956260681,0.9411839842796326,0.014939365085478117,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,128,0.375,1.0,1.0,0.5703125,0.0,1.118033988749895,5.0,6.008641773518898,0.580810546875,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,1.2796800136566162,1.3484159708023071,1.3006976008415223,1.2929120063781738,0.020807998177176254,32,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,256,0.375,2.0,2.0,0.8828125,0.0,0.6959705453537527,3.0,6.60872850615583,0.38055419921875,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,1.3630399703979492,1.4430400133132935,1.3909215927124023,1.3892319798469543,0.022335744492366926,64,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,512,0.375,4.0,4.0,0.984375,0.0,0.5201036555341637,3.0,6.798826509158851,0.28302001953125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,1.27948796749115,1.3904000520706177,1.3176063895225525,1.309440016746521,0.038060887827312775,128,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,1024,0.375,8.0,8.0,1.0,0.25,0.3511282039725661,1.875,6.91002266305238,0.1970977783203125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,1.264415979385376,1.3145920038223267,1.2791999936103822,1.2753440141677856,0.014130605249568332,256,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,2048,0.375,16.0,16.0,1.0,0.375,0.24692938483248605,1.6875,6.955481130775285,0.13909912109375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,1.3081920146942139,1.347648024559021,1.3292255997657776,1.329967975616455,0.014558863679016933,512,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,4096,0.375,32.0,32.0,1.0,0.625,0.17143053326165383,1.5625,6.9786675275754035,0.09520339965820312,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,1.242751955986023,1.3112000226974487,1.2747935891151427,1.266207993030548,0.021093073517695057,1024,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,8192,0.375,64.0,64.0,1.0,0.78125,0.11000099875256815,1.296875,6.991308871213679,0.062183380126953125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,1.7388160228729248,1.8077759742736816,1.772764801979065,1.772704005241394,0.021056644077284283,2048,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,16384,0.375,128.0,128.0,1.0,0.78125,0.0864630150197678,1.1796875,6.994552526394139,0.048796653747558594,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,2.6563520431518555,2.7063679695129395,2.6785055875778196,2.6791679859161377,0.01639963463052944,4096,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,32768,0.375,256.0,256.0,1.0,0.8671875,0.06127686514721937,1.16015625,6.997291583027146,0.03497934341430664,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,4.386879920959473,4.452256202697754,4.4108480453491214,4.406303882598877,0.019768161791937636,8192,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,65536,0.375,512.0,512.0,1.0,0.884765625,0.041723768525324195,1.1171875,6.998746434318934,0.02298593521118164,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.24208000302314758,0.4028480052947998,0.3041536003351212,0.277103990316391,0.05660721484881584,1,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,0.2447039932012558,0.30502399802207947,0.26446720361709597,0.26265600323677063,0.016744548364435372,8,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,64,0.375,0.5,0.5,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,0.2337920069694519,0.2881599962711334,0.26074880361557007,0.264384001493454,0.016469850143940968,16,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,128,0.375,1.0,1.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,0.23369599878787994,0.28591999411582947,0.25465920120477675,0.25385600328445435,0.01593204652619194,32,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,256,0.375,2.0,2.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,0.2295999974012375,0.26556798815727234,0.24674240052700042,0.2497600018978119,0.010345732066199003,64,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,512,0.375,4.0,4.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,0.21721599996089935,0.29020801186561584,0.2394208014011383,0.2346400022506714,0.018747330357768585,128,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,1024,0.375,8.0,8.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,0.2717759907245636,0.305184006690979,0.28813759982585907,0.28809599578380585,0.01183422600545854,256,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,2048,0.375,16.0,16.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,0.3917759954929352,0.43772798776626587,0.41130879521369934,0.4131519943475723,0.012992473640805227,512,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,4096,0.375,32.0,32.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,0.6176639795303345,0.7009919881820679,0.642767995595932,0.6330719888210297,0.024074084919938756,1024,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,8192,0.375,64.0,64.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,1.0820800065994263,1.1674879789352417,1.1034304022789,1.0977439880371094,0.0233067292981566,2048,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,16384,0.375,128.0,128.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,1.9809919595718384,2.0415360927581787,2.003715181350708,1.992751955986023,0.022066076434645737,4096,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,32768,0.375,256.0,256.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,3.790112018585205,3.8651199340820312,3.829139161109924,3.8230879306793213,0.025177464160110564,8192,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,65536,0.375,512.0,512.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.212351992726326,0.24383999407291412,0.22760000079870224,0.22723200172185898,0.01050568575837594,1,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,0.47494399547576904,0.5184000134468079,0.4920704007148743,0.49169600009918213,0.011991064701471855,8,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,64,0.375,0.5,0.5,0.3984375,0.0,1.3919410907075054,6.0,5.570159765557392,0.667236328125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,0.6360960006713867,0.7004479765892029,0.6608384013175964,0.6572319865226746,0.020416877242438597,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,128,0.375,1.0,1.0,0.625,0.0,1.0307764064044151,4.0,6.138251855282827,0.5382080078125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,0.780896008014679,0.8301439881324768,0.80346559882164,0.8030399978160858,0.016801230312128875,32,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,256,0.375,2.0,2.0,0.859375,0.0,0.6903350635742038,3.0,6.5943747091218174,0.38067626953125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,0.8607040047645569,0.9195200204849243,0.8783008038997651,0.8751039803028107,0.01719059253115595,64,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,512,0.375,4.0,4.0,0.9765625,0.0,0.49410588440130926,2.75,6.814452474347134,0.271270751953125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,0.833952009677887,0.894752025604248,0.8619967997074127,0.863215982913971,0.018716378851797198,128,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,1024,0.375,8.0,8.0,1.0,0.25,0.33693529145074724,2.125,6.9186075263155535,0.1867218017578125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,0.834879994392395,0.8871039748191833,0.8651552021503448,0.8638879954814911,0.015262430894400969,256,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,2048,0.375,16.0,16.0,1.0,0.375,0.25567294018677456,1.8125,6.952441049154937,0.14349365234375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,0.8518080115318298,0.9097599983215332,0.8810272097587586,0.8751040101051331,0.017005819633271906,512,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,4096,0.375,32.0,32.0,1.0,0.625,0.1747801353218523,1.53125,6.978069554482723,0.09820938110351562,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,0.8470079898834229,0.9010239839553833,0.8694015920162201,0.8716959953308105,0.016138881171190216,1024,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,8192,0.375,64.0,64.0,1.0,0.65625,0.1158122428154187,1.3125,6.9901908183358845,0.06445503234863281,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,1.1698240041732788,1.2311359643936157,1.1888479948043824,1.1890720129013062,0.017978797794492758,2048,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,16384,0.375,128.0,128.0,1.0,0.78125,0.08347181893108634,1.1796875,6.994921772573154,0.046871185302734375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,1.7702720165252686,1.8097599744796753,1.7919103980064393,1.7956640124320984,0.012641295676021557,4096,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,32768,0.375,256.0,256.0,1.0,0.78125,0.06866734477822484,1.20703125,6.996602562938728,0.03801727294921875,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,2.968672037124634,3.0278079509735107,2.9899007797241213,2.9824799299240112,0.01816414122631667,8192,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,65536,0.375,512.0,512.0,1.0,0.8984375,0.04399546833977376,1.126953125,6.998607314922362,0.024699926376342773,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.19574399292469025,0.2512960135936737,0.21939200013875962,0.2199999988079071,0.017532156418212565,1,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,0.20483200252056122,0.24006399512290955,0.22215040028095245,0.22433599829673767,0.00969639786132892,8,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,64,0.375,0.5,0.5,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,0.20559999346733093,0.24726399779319763,0.22126719802618028,0.22207999974489212,0.01328093478485499,16,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,128,0.375,1.0,1.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,0.20003199577331543,0.2301120012998581,0.21453119963407516,0.21598400175571442,0.010402239855151332,32,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,256,0.375,2.0,2.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,0.19551999866962433,0.22972799837589264,0.21238719969987868,0.21488000452518463,0.010706095835489097,64,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,512,0.375,4.0,4.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,0.19420799612998962,0.2903999984264374,0.2211231991648674,0.21476799994707108,0.025955008886196004,128,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,1024,0.375,8.0,8.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,0.2290560007095337,0.289247989654541,0.2543327987194061,0.24939200282096863,0.01663236753503115,256,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,2048,0.375,16.0,16.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,0.30831998586654663,0.36953601241111755,0.3324000000953674,0.3288639932870865,0.018617999572156707,512,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,4096,0.375,32.0,32.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,0.462911993265152,0.5497919917106628,0.4893856018781662,0.4816960096359253,0.023348152887178286,1024,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,8192,0.375,64.0,64.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,0.7662079930305481,0.8717759847640991,0.788454395532608,0.7744799852371216,0.03174746482604812,2048,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,16384,0.375,128.0,128.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,1.363935947418213,1.4143040180206299,1.3812703967094422,1.3798720240592957,0.014770450075530007,4096,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,32768,0.375,256.0,256.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,2.5507519245147705,2.680704116821289,2.579859209060669,2.566223978996277,0.03673283558365955,8192,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,65536,0.375,512.0,512.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/artifacts.sha256

@@ -0,0 +1,8 @@
+3937c623adcea3d379b82bb7ab63d3b293f917fa59fa6fd6a147581d6a252cb6  runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/raw/flashattn-code-longctx-tp1.json
+4e6d6002db037bdc71d40f76d6de979ef47960612c15dd985c04048c25bd1d41  runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/aituner.commit
+420e4a613b1bcdb44b873eaa911f4a23129713ff9d486fc783b31a009a55cc85  runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/batch-specs.txt
+5dd612a7806900e9d12afc27ccef196cc5518aea831aa4ed3b0a53bcfb5746cf  runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/max-model-len.txt
+484da176e7355ab5532d5129a8e78bea0ae84b3aa0a42c7f0bea9689667762ff  runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/pip-freeze.txt
+595b25965f76bf7ac25aa3b60126a437a083619303303aaf63346e993862253a  runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/source.sha256
+4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865  runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/vllm-allow-long-max-model-len.txt
+11e8f5af440e3db4c01b519a7b0e30bbcecc3a57bfb53dd86d32e601beb95a32  runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/vllm-source.commit

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/aituner.commit

@@ -0,0 +1 @@
+e251046c306a8e03214fe1b2f40caabb98f76b7b

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/batch-specs.txt

@@ -0,0 +1,13 @@
+q8ks48k
+q8ks64k
+q8ks80k
+q8ks96k
+q8ks112k
+q8ks128k
+q8ks136k
+q512s128k
+q2ks66k
+q4ks100k
+q6ks134k
+q1ks8k
+q512s4k

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/max-model-len.txt

@@ -0,0 +1 @@
+147456

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/pip-freeze.txt

@@ -0,0 +1,179 @@
+aiohappyeyeballs==2.7.1
+aiohttp==3.14.1
+aiosignal==1.4.0
+annotated-doc==0.0.4
+annotated-types==0.7.0
+anthropic==0.117.0
+anyio==4.14.2
+apache-tvm-ffi==0.1.9
+astor==0.8.1
+attrs==26.1.0
+blake3==1.0.9
+cachetools==7.1.4
+cbor2==6.1.3
+certifi==2026.6.17
+cffi==2.1.0
+charset-normalizer==3.4.9
+click==8.4.2
+cloudpickle==3.1.2
+compressed-tensors==0.15.0.1
+cryptography==49.0.0
+cuda-bindings==12.9.7
+cuda-pathfinder==1.5.6
+cuda-python==12.9.7
+cuda-tile==1.5.0
+cuda-toolkit==12.9.1
+depyf==0.20.0
+detect-installer==0.1.0
+dill==0.4.1
+diskcache==5.6.3
+distro==1.9.0
+dnspython==2.8.0
+docstring-parser==0.18.0
+einops==0.8.2
+email-validator==2.3.0
+fastapi==0.139.2
+fastapi-cli==0.0.32
+fastapi-cloud-cli==0.22.2
+fastar==0.11.0
+fastsafetensors==0.3.3
+filelock==3.31.1
+flashinfer-cubin==0.6.8.post1
+flashinfer-python==0.6.8.post1
+frozenlist==1.8.0
+fsspec==2026.6.0
+gguf==0.19.0
+googleapis-common-protos==1.75.0
+grpcio==1.82.1
+h11==0.16.0
+hf-xet==1.5.2
+httpcore==1.0.9
+httptools==0.8.0
+httpx==0.28.1
+httpx-sse==0.4.3
+huggingface-hub==1.24.0
+idna==3.18
+ijson==3.5.1
+interegular==0.3.3
+jinja2==3.1.6
+jiter==0.16.0
+jmespath==1.1.0
+jsonschema==4.26.0
+jsonschema-specifications==2025.9.1
+lark==1.2.2
+llguidance==1.3.0
+llvmlite==0.47.0
+lm-format-enforcer==0.11.3
+loguru==0.7.3
+markdown-it-py==4.2.0
+markupsafe==3.0.3
+mcp==1.28.1
+mdurl==0.1.2
+mistral-common==1.11.6
+ml-dtypes==0.5.4
+model-hosting-container-standards==0.1.16
+mpmath==1.3.0
+msgspec==0.21.1
+multidict==6.7.1
+networkx==3.6.1
+ninja==1.13.0
+numba==0.65.0
+numpy==2.3.5
+nvidia-cublas-cu12==12.9.1.4
+nvidia-cuda-cupti-cu12==12.9.79
+nvidia-cuda-nvrtc-cu12==12.9.86
+nvidia-cuda-runtime-cu12==12.9.79
+nvidia-cudnn-cu12==9.17.1.4
+nvidia-cudnn-frontend==1.18.0
+nvidia-cufft-cu12==11.4.1.4
+nvidia-cufile-cu12==1.14.1.1
+nvidia-curand-cu12==10.3.10.19
+nvidia-cusolver-cu12==11.7.5.82
+nvidia-cusparse-cu12==12.5.10.65
+nvidia-cusparselt-cu12==0.7.1
+nvidia-cutlass-dsl==4.5.3
+nvidia-cutlass-dsl-libs-base==4.5.3
+nvidia-ml-py==13.610.43
+nvidia-nccl-cu12==2.28.9
+nvidia-nvjitlink-cu12==12.9.86
+nvidia-nvshmem-cu12==3.4.5
+nvidia-nvtx-cu12==12.9.79
+openai==2.46.0
+openai-harmony==0.0.8
+opencv-python-headless==5.0.0.93
+opentelemetry-api==1.44.0
+opentelemetry-exporter-otlp==1.44.0
+opentelemetry-exporter-otlp-proto-common==1.44.0
+opentelemetry-exporter-otlp-proto-grpc==1.44.0
+opentelemetry-exporter-otlp-proto-http==1.44.0
+opentelemetry-proto==1.44.0
+opentelemetry-sdk==1.44.0
+opentelemetry-semantic-conventions==0.65b0
+opentelemetry-semantic-conventions-ai==0.5.1
+outlines-core==0.2.14
+packaging==26.2
+partial-json-parser==0.2.1.1.post7
+pillow==12.3.0
+prometheus-client==0.25.0
+prometheus-fastapi-instrumentator==8.0.2
+propcache==0.5.2
+protobuf==7.35.1
+psutil==7.2.2
+py-cpuinfo==9.0.0
+pybase64==1.4.3
+pycountry==26.2.16
+pycparser==3.0
+pydantic==2.13.4
+pydantic-core==2.46.4
+pydantic-extra-types==2.11.1
+pydantic-settings==2.14.2
+pygments==2.20.0
+pyjwt==2.13.0
+python-dotenv==1.2.2
+python-json-logger==4.1.0
+python-multipart==0.0.32
+pyyaml==6.0.3
+pyzmq==27.1.0
+quack-kernels==0.5.0
+referencing==0.37.0
+regex==2026.7.19
+requests==2.34.2
+rich==15.0.0
+rich-toolkit==0.20.3
+rignore==0.8.0
+rpds-py==2026.6.3
+safetensors==0.8.0
+sentencepiece==0.2.2
+sentry-sdk==2.66.0
+setproctitle==1.3.7
+setuptools==80.10.2
+shellingham==1.5.4
+six==1.17.0
+sniffio==1.3.1
+sse-starlette==3.4.5
+starlette==1.3.1
+supervisor==4.3.0
+sympy==1.14.0
+tabulate==0.10.0
+tiktoken==0.13.0
+tilelang==0.1.9
+tokenizers==0.22.2
+torch==2.11.0+cu129
+torch-c-dlpack-ext==0.1.5
+torchaudio==2.11.0+cu129
+torchvision==0.26.0+cu129
+tqdm==4.69.0
+transformers==5.14.1
+triton==3.6.0
+typer==0.27.0
+typing-extensions==4.16.0
+typing-inspection==0.4.2
+urllib3==2.7.0
+uvicorn==0.51.0
+uvloop==0.22.1
+vllm @ https://github.com/vllm-project/vllm/releases/download/v0.20.0/vllm-0.20.0%2Bcu129-cp38-abi3-manylinux_2_31_x86_64.whl
+watchfiles==1.2.0
+websockets==16.1.1
+xgrammar==0.2.3
+yarl==1.24.5
+z3-solver==4.15.4.0

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/source.sha256

@@ -0,0 +1,2 @@
+42381f4bd1b67710d4068eb8993240930326a804c6e5167b0b78df3420e47c74  /home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/../frontier-qwen30-vllm020-profile-v1/profile_vllm020_flashattn.py
+8a7a72b3adb25db6a61a86f9b04d2d19ebbfe3f2d20480852ef2f45376ac3b85  runs/frontier-code-trace-v0/run_flashattn_code_longctx.sh

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/vllm-allow-long-max-model-len.txt

@@ -0,0 +1 @@
+1

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/provenance/vllm-source.commit

@@ -0,0 +1 @@
+88d34c6409e9fb3c7b8ca0c04756f061d2099eb1

runs/frontier-code-trace-v0/profiles/raw/tp1-r1-v2/raw/flashattn-code-longctx-tp1.json

@@ -0,0 +1,564 @@
+{
+  "environment": {
+    "attention_backend": "FLASH_ATTN",
+    "block_size": 16,
+    "dtype": "bfloat16",
+    "gpu": "NVIDIA H20",
+    "model": "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B",
+    "profile_kv_update": true,
+    "profile_method": "cuda_event",
+    "torch_cuda": "12.9",
+    "torch_version": "2.11.0+cu129",
+    "vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1",
+    "vllm_version": "0.20.0"
+  },
+  "rows": [
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks48k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.0851840004324913,
+        "mean_ms": 0.07839040011167527,
+        "median_ms": 0.07703999802470207,
+        "min_ms": 0.07552000135183334,
+        "std_ms": 0.0030805904904383677
+      },
+      "max_time": 0.043892990112304686,
+      "mean_time": 0.04368390693664551,
+      "memory_allocated_mb": 240.0849609375,
+      "memory_reserved_mb": 246.0,
+      "min_time": 0.043620254516601564,
+      "std_time": 8.458163192147439e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 187529.01410308387
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks64k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.0854400023818016,
+        "mean_ms": 0.07895359992980958,
+        "median_ms": 0.07791999727487564,
+        "min_ms": 0.07718399912118912,
+        "std_ms": 0.0024425064759109735
+      },
+      "max_time": 0.059730175018310544,
+      "mean_time": 0.0595021312713623,
+      "memory_allocated_mb": 272.0888671875,
+      "memory_reserved_mb": 374.0,
+      "min_time": 0.05942179107666016,
+      "std_time": 0.00010367381308510448,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 137675.74076699864
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks80k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.09055999666452408,
+        "mean_ms": 0.08208959847688675,
+        "median_ms": 0.08087999746203423,
+        "min_ms": 0.07875200361013412,
+        "std_ms": 0.0033614630055883482
+      },
+      "max_time": 0.07555481719970703,
+      "mean_time": 0.07538758392333984,
+      "memory_allocated_mb": 304.0927734375,
+      "memory_reserved_mb": 534.0,
+      "min_time": 0.0752852783203125,
+      "std_time": 8.870109119325343e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 108665.10867797918
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks96k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.09888000041246414,
+        "mean_ms": 0.0819871999323368,
+        "median_ms": 0.08008000254631042,
+        "min_ms": 0.07788799703121185,
+        "std_ms": 0.005835618489111142
+      },
+      "max_time": 0.09145164489746094,
+      "mean_time": 0.09124225158691406,
+      "memory_allocated_mb": 336.0966796875,
+      "memory_reserved_mb": 726.0,
+      "min_time": 0.09113442993164063,
+      "std_time": 0.0001009018902177725,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 89782.96630697016
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks112k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08790399879217148,
+        "mean_ms": 0.07967040091753005,
+        "median_ms": 0.07808000221848488,
+        "min_ms": 0.07596799731254578,
+        "std_ms": 0.0036209252740976605
+      },
+      "max_time": 0.10734063720703126,
+      "mean_time": 0.1069560287475586,
+      "memory_allocated_mb": 368.1005859375,
+      "memory_reserved_mb": 950.0,
+      "min_time": 0.10689055633544922,
+      "std_time": 0.00013032874360676438,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 76592.22295299545
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08534400165081024,
+        "mean_ms": 0.07928640022873878,
+        "median_ms": 0.07787200063467026,
+        "min_ms": 0.07648000121116638,
+        "std_ms": 0.0028305104107121735
+      },
+      "max_time": 0.1229840316772461,
+      "mean_time": 0.12283342437744141,
+      "memory_allocated_mb": 400.1044921875,
+      "memory_reserved_mb": 1206.0,
+      "min_time": 0.12271724700927734,
+      "std_time": 9.625274868603512e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 66691.94514049937
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks136k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08774399757385254,
+        "mean_ms": 0.08081279993057251,
+        "median_ms": 0.07972799986600876,
+        "min_ms": 0.07664000242948532,
+        "std_ms": 0.003487270970977775
+      },
+      "max_time": 0.13109359741210938,
+      "mean_time": 0.13079229431152345,
+      "memory_allocated_mb": 416.1064453125,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.1306565399169922,
+      "std_time": 0.00013138911961272913,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 62633.65929255852
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.040063999593257904,
+        "mean_ms": 0.019459199998527764,
+        "median_ms": 0.01601599995046854,
+        "min_ms": 0.01532800029963255,
+        "std_ms": 0.007496165774486326
+      },
+      "max_time": 0.009649087905883789,
+      "mean_time": 0.009489968109130859,
+      "memory_allocated_mb": 265.0458984375,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.009443936347961425,
+      "std_time": 5.961646816296781e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 53951.70922728124
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q2ks66k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.040832001715898514,
+        "mean_ms": 0.02942080032080412,
+        "median_ms": 0.027888000011444092,
+        "min_ms": 0.02691200003027916,
+        "std_ms": 0.004040048279091587
+      },
+      "max_time": 0.016906623840332032,
+      "mean_time": 0.01678766403198242,
+      "memory_allocated_mb": 168.04248046875,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.016753759384155274,
+      "std_time": 4.236470233451937e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 121994.34037387963
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q4ks100k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.04867200180888176,
+        "mean_ms": 0.04565120078623295,
+        "median_ms": 0.04468800127506256,
+        "min_ms": 0.04416000097990036,
+        "std_ms": 0.0018079971851529544
+      },
+      "max_time": 0.05050300979614258,
+      "mean_time": 0.05035054740905761,
+      "memory_allocated_mb": 272.06640625,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.05029715347290039,
+      "std_time": 7.24480602714254e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 81349.66173700758
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q6ks134k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.06684800237417221,
+        "mean_ms": 0.06276160031557083,
+        "median_ms": 0.061824001371860504,
+        "min_ms": 0.060447998344898224,
+        "std_ms": 0.0024045636532566625
+      },
+      "max_time": 0.09653209686279297,
+      "mean_time": 0.0961898666381836,
+      "memory_allocated_mb": 376.09033203125,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.09607142639160156,
+      "std_time": 0.0001365794879996899,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 63873.67208970714
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q1ks8k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.0326399989426136,
+        "mean_ms": 0.021955199912190436,
+        "median_ms": 0.020560000091791153,
+        "min_ms": 0.018559999763965607,
+        "std_ms": 0.00393975597463501
+      },
+      "max_time": 0.0011526399850845337,
+      "mean_time": 0.0011425888061523438,
+      "memory_allocated_mb": 34.0205078125,
+      "memory_reserved_mb": 1480.0,
+      "min_time": 0.0011403199434280396,
+      "std_time": 3.498447348453724e-06,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 896210.4253833097
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s4k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.02783999964594841,
+        "mean_ms": 0.017366399988532066,
+        "median_ms": 0.01611199975013733,
+        "min_ms": 0.015424000099301338,
+        "std_ms": 0.003547472261377056
+      },
+      "max_time": 0.00034601598978042605,
+      "mean_time": 0.0003366623997688293,
+      "memory_allocated_mb": 17.015625,
+      "memory_reserved_mb": 1480.0,
+      "min_time": 0.0003308480083942413,
+      "std_time": 4.813544996936389e-06,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 1520811.3539010207
+    }
+  ],
+  "schema_version": "qwen30_vllm020_flashattn_raw.v1"
+}

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/artifacts.sha256

@@ -0,0 +1,8 @@
+f113ab5a05167f8ac4c1938c4a04fadc11d5dc2bb8c536a69db23638e6ef73d4  runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/raw/flashattn-code-longctx-tp1.json
+4e6d6002db037bdc71d40f76d6de979ef47960612c15dd985c04048c25bd1d41  runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/aituner.commit
+420e4a613b1bcdb44b873eaa911f4a23129713ff9d486fc783b31a009a55cc85  runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/batch-specs.txt
+5dd612a7806900e9d12afc27ccef196cc5518aea831aa4ed3b0a53bcfb5746cf  runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/max-model-len.txt
+484da176e7355ab5532d5129a8e78bea0ae84b3aa0a42c7f0bea9689667762ff  runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/pip-freeze.txt
+595b25965f76bf7ac25aa3b60126a437a083619303303aaf63346e993862253a  runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/source.sha256
+4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865  runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/vllm-allow-long-max-model-len.txt
+11e8f5af440e3db4c01b519a7b0e30bbcecc3a57bfb53dd86d32e601beb95a32  runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/vllm-source.commit

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/aituner.commit

@@ -0,0 +1 @@
+e251046c306a8e03214fe1b2f40caabb98f76b7b

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/batch-specs.txt

@@ -0,0 +1,13 @@
+q8ks48k
+q8ks64k
+q8ks80k
+q8ks96k
+q8ks112k
+q8ks128k
+q8ks136k
+q512s128k
+q2ks66k
+q4ks100k
+q6ks134k
+q1ks8k
+q512s4k

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/max-model-len.txt

@@ -0,0 +1 @@
+147456

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/pip-freeze.txt

@@ -0,0 +1,179 @@
+aiohappyeyeballs==2.7.1
+aiohttp==3.14.1
+aiosignal==1.4.0
+annotated-doc==0.0.4
+annotated-types==0.7.0
+anthropic==0.117.0
+anyio==4.14.2
+apache-tvm-ffi==0.1.9
+astor==0.8.1
+attrs==26.1.0
+blake3==1.0.9
+cachetools==7.1.4
+cbor2==6.1.3
+certifi==2026.6.17
+cffi==2.1.0
+charset-normalizer==3.4.9
+click==8.4.2
+cloudpickle==3.1.2
+compressed-tensors==0.15.0.1
+cryptography==49.0.0
+cuda-bindings==12.9.7
+cuda-pathfinder==1.5.6
+cuda-python==12.9.7
+cuda-tile==1.5.0
+cuda-toolkit==12.9.1
+depyf==0.20.0
+detect-installer==0.1.0
+dill==0.4.1
+diskcache==5.6.3
+distro==1.9.0
+dnspython==2.8.0
+docstring-parser==0.18.0
+einops==0.8.2
+email-validator==2.3.0
+fastapi==0.139.2
+fastapi-cli==0.0.32
+fastapi-cloud-cli==0.22.2
+fastar==0.11.0
+fastsafetensors==0.3.3
+filelock==3.31.1
+flashinfer-cubin==0.6.8.post1
+flashinfer-python==0.6.8.post1
+frozenlist==1.8.0
+fsspec==2026.6.0
+gguf==0.19.0
+googleapis-common-protos==1.75.0
+grpcio==1.82.1
+h11==0.16.0
+hf-xet==1.5.2
+httpcore==1.0.9
+httptools==0.8.0
+httpx==0.28.1
+httpx-sse==0.4.3
+huggingface-hub==1.24.0
+idna==3.18
+ijson==3.5.1
+interegular==0.3.3
+jinja2==3.1.6
+jiter==0.16.0
+jmespath==1.1.0
+jsonschema==4.26.0
+jsonschema-specifications==2025.9.1
+lark==1.2.2
+llguidance==1.3.0
+llvmlite==0.47.0
+lm-format-enforcer==0.11.3
+loguru==0.7.3
+markdown-it-py==4.2.0
+markupsafe==3.0.3
+mcp==1.28.1
+mdurl==0.1.2
+mistral-common==1.11.6
+ml-dtypes==0.5.4
+model-hosting-container-standards==0.1.16
+mpmath==1.3.0
+msgspec==0.21.1
+multidict==6.7.1
+networkx==3.6.1
+ninja==1.13.0
+numba==0.65.0
+numpy==2.3.5
+nvidia-cublas-cu12==12.9.1.4
+nvidia-cuda-cupti-cu12==12.9.79
+nvidia-cuda-nvrtc-cu12==12.9.86
+nvidia-cuda-runtime-cu12==12.9.79
+nvidia-cudnn-cu12==9.17.1.4
+nvidia-cudnn-frontend==1.18.0
+nvidia-cufft-cu12==11.4.1.4
+nvidia-cufile-cu12==1.14.1.1
+nvidia-curand-cu12==10.3.10.19
+nvidia-cusolver-cu12==11.7.5.82
+nvidia-cusparse-cu12==12.5.10.65
+nvidia-cusparselt-cu12==0.7.1
+nvidia-cutlass-dsl==4.5.3
+nvidia-cutlass-dsl-libs-base==4.5.3
+nvidia-ml-py==13.610.43
+nvidia-nccl-cu12==2.28.9
+nvidia-nvjitlink-cu12==12.9.86
+nvidia-nvshmem-cu12==3.4.5
+nvidia-nvtx-cu12==12.9.79
+openai==2.46.0
+openai-harmony==0.0.8
+opencv-python-headless==5.0.0.93
+opentelemetry-api==1.44.0
+opentelemetry-exporter-otlp==1.44.0
+opentelemetry-exporter-otlp-proto-common==1.44.0
+opentelemetry-exporter-otlp-proto-grpc==1.44.0
+opentelemetry-exporter-otlp-proto-http==1.44.0
+opentelemetry-proto==1.44.0
+opentelemetry-sdk==1.44.0
+opentelemetry-semantic-conventions==0.65b0
+opentelemetry-semantic-conventions-ai==0.5.1
+outlines-core==0.2.14
+packaging==26.2
+partial-json-parser==0.2.1.1.post7
+pillow==12.3.0
+prometheus-client==0.25.0
+prometheus-fastapi-instrumentator==8.0.2
+propcache==0.5.2
+protobuf==7.35.1
+psutil==7.2.2
+py-cpuinfo==9.0.0
+pybase64==1.4.3
+pycountry==26.2.16
+pycparser==3.0
+pydantic==2.13.4
+pydantic-core==2.46.4
+pydantic-extra-types==2.11.1
+pydantic-settings==2.14.2
+pygments==2.20.0
+pyjwt==2.13.0
+python-dotenv==1.2.2
+python-json-logger==4.1.0
+python-multipart==0.0.32
+pyyaml==6.0.3
+pyzmq==27.1.0
+quack-kernels==0.5.0
+referencing==0.37.0
+regex==2026.7.19
+requests==2.34.2
+rich==15.0.0
+rich-toolkit==0.20.3
+rignore==0.8.0
+rpds-py==2026.6.3
+safetensors==0.8.0
+sentencepiece==0.2.2
+sentry-sdk==2.66.0
+setproctitle==1.3.7
+setuptools==80.10.2
+shellingham==1.5.4
+six==1.17.0
+sniffio==1.3.1
+sse-starlette==3.4.5
+starlette==1.3.1
+supervisor==4.3.0
+sympy==1.14.0
+tabulate==0.10.0
+tiktoken==0.13.0
+tilelang==0.1.9
+tokenizers==0.22.2
+torch==2.11.0+cu129
+torch-c-dlpack-ext==0.1.5
+torchaudio==2.11.0+cu129
+torchvision==0.26.0+cu129
+tqdm==4.69.0
+transformers==5.14.1
+triton==3.6.0
+typer==0.27.0
+typing-extensions==4.16.0
+typing-inspection==0.4.2
+urllib3==2.7.0
+uvicorn==0.51.0
+uvloop==0.22.1
+vllm @ https://github.com/vllm-project/vllm/releases/download/v0.20.0/vllm-0.20.0%2Bcu129-cp38-abi3-manylinux_2_31_x86_64.whl
+watchfiles==1.2.0
+websockets==16.1.1
+xgrammar==0.2.3
+yarl==1.24.5
+z3-solver==4.15.4.0

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/source.sha256

@@ -0,0 +1,2 @@
+42381f4bd1b67710d4068eb8993240930326a804c6e5167b0b78df3420e47c74  /home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/../frontier-qwen30-vllm020-profile-v1/profile_vllm020_flashattn.py
+8a7a72b3adb25db6a61a86f9b04d2d19ebbfe3f2d20480852ef2f45376ac3b85  runs/frontier-code-trace-v0/run_flashattn_code_longctx.sh

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/vllm-allow-long-max-model-len.txt

@@ -0,0 +1 @@
+1

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/provenance/vllm-source.commit

@@ -0,0 +1 @@
+88d34c6409e9fb3c7b8ca0c04756f061d2099eb1

runs/frontier-code-trace-v0/profiles/raw/tp1-r2-v2/raw/flashattn-code-longctx-tp1.json

@@ -0,0 +1,564 @@
+{
+  "environment": {
+    "attention_backend": "FLASH_ATTN",
+    "block_size": 16,
+    "dtype": "bfloat16",
+    "gpu": "NVIDIA H20",
+    "model": "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B",
+    "profile_kv_update": true,
+    "profile_method": "cuda_event",
+    "torch_cuda": "12.9",
+    "torch_version": "2.11.0+cu129",
+    "vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1",
+    "vllm_version": "0.20.0"
+  },
+  "rows": [
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks48k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08799999952316284,
+        "mean_ms": 0.08035840019583702,
+        "median_ms": 0.08003199845552444,
+        "min_ms": 0.07583999633789062,
+        "std_ms": 0.003823532942228132
+      },
+      "max_time": 0.04384675216674805,
+      "mean_time": 0.043662386703491214,
+      "memory_allocated_mb": 240.0849609375,
+      "memory_reserved_mb": 246.0,
+      "min_time": 0.04360464096069336,
+      "std_time": 7.18445670309503e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 187621.4430427591
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks64k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08611200004816055,
+        "mean_ms": 0.07973119989037514,
+        "median_ms": 0.07807999849319458,
+        "min_ms": 0.07692799717187881,
+        "std_ms": 0.0030488875049308785
+      },
+      "max_time": 0.05954601669311523,
+      "mean_time": 0.0594769702911377,
+      "memory_allocated_mb": 272.0888671875,
+      "memory_reserved_mb": 374.0,
+      "min_time": 0.05941872024536133,
+      "std_time": 4.5624842215119615e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 137733.98274828802
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks80k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.09087999910116196,
+        "mean_ms": 0.08214399963617325,
+        "median_ms": 0.08115199953317642,
+        "min_ms": 0.07846400141716003,
+        "std_ms": 0.003409163930931844
+      },
+      "max_time": 0.07549359893798828,
+      "mean_time": 0.07529304885864258,
+      "memory_allocated_mb": 304.0927734375,
+      "memory_reserved_mb": 534.0,
+      "min_time": 0.07523104095458985,
+      "std_time": 8.131664708395529e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 108801.5444211843
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks96k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08684799820184708,
+        "mean_ms": 0.08159359991550445,
+        "median_ms": 0.08057600259780884,
+        "min_ms": 0.0790719985961914,
+        "std_ms": 0.0025743110833498623
+      },
+      "max_time": 0.09123193359375,
+      "mean_time": 0.09110229187011719,
+      "memory_allocated_mb": 336.0966796875,
+      "memory_reserved_mb": 726.0,
+      "min_time": 0.09105142211914062,
+      "std_time": 5.304259171893706e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 89920.89915453696
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks112k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08508799970149994,
+        "mean_ms": 0.07919999957084656,
+        "median_ms": 0.0780319981276989,
+        "min_ms": 0.0764480009675026,
+        "std_ms": 0.002612344815356456
+      },
+      "max_time": 0.10705843353271484,
+      "mean_time": 0.10692193984985351,
+      "memory_allocated_mb": 368.1005859375,
+      "memory_reserved_mb": 950.0,
+      "min_time": 0.10688256072998047,
+      "std_time": 5.1132907623503757e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 76616.64211763947
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08479999750852585,
+        "mean_ms": 0.07948800027370453,
+        "median_ms": 0.07824000343680382,
+        "min_ms": 0.0764160007238388,
+        "std_ms": 0.0028062370889137406
+      },
+      "max_time": 0.12304061126708984,
+      "mean_time": 0.12278123931884766,
+      "memory_allocated_mb": 400.1044921875,
+      "memory_reserved_mb": 1206.0,
+      "min_time": 0.122716796875,
+      "std_time": 9.14188387039285e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 66720.29086403332
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks136k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.08329600095748901,
+        "mean_ms": 0.07985600009560585,
+        "median_ms": 0.07945600152015686,
+        "min_ms": 0.07705599814653397,
+        "std_ms": 0.0021785520776957702
+      },
+      "max_time": 0.13100553894042968,
+      "mean_time": 0.1307180953979492,
+      "memory_allocated_mb": 416.1064453125,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.1306175994873047,
+      "std_time": 0.000107931256378197,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 62669.21174961153
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.02659199945628643,
+        "mean_ms": 0.01834559990093112,
+        "median_ms": 0.01649599988013506,
+        "min_ms": 0.015135999768972397,
+        "std_ms": 0.00375367086613964
+      },
+      "max_time": 0.009480863571166993,
+      "mean_time": 0.009440905380249024,
+      "memory_allocated_mb": 265.0458984375,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.009432319641113282,
+      "std_time": 1.4279626944428254e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 54232.08679446535
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q2ks66k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.0331839993596077,
+        "mean_ms": 0.028758399933576585,
+        "median_ms": 0.027872000820934772,
+        "min_ms": 0.026623999699950218,
+        "std_ms": 0.002088649186504863
+      },
+      "max_time": 0.01684105682373047,
+      "mean_time": 0.016775878524780276,
+      "memory_allocated_mb": 168.04248046875,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.016756032943725584,
+      "std_time": 2.3830672818836446e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 122080.04468885626
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q4ks100k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.052480001002550125,
+        "mean_ms": 0.045238399878144264,
+        "median_ms": 0.04420800134539604,
+        "min_ms": 0.04303999990224838,
+        "std_ms": 0.0027615853312914604
+      },
+      "max_time": 0.050535968780517575,
+      "mean_time": 0.0503309886932373,
+      "memory_allocated_mb": 272.06640625,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.05029235076904297,
+      "std_time": 6.98791171184278e-05,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 81381.27436686649
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q6ks134k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.0740479975938797,
+        "mean_ms": 0.06334079951047897,
+        "median_ms": 0.06135999970138073,
+        "min_ms": 0.060095999389886856,
+        "std_ms": 0.004142020833679573
+      },
+      "max_time": 0.09652816009521484,
+      "mean_time": 0.09613609313964844,
+      "memory_allocated_mb": 376.09033203125,
+      "memory_reserved_mb": 1478.0,
+      "min_time": 0.09607071685791016,
+      "std_time": 0.0001328159156966631,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 63909.39967859056
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q1ks8k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.028192000463604927,
+        "mean_ms": 0.021113600023090838,
+        "median_ms": 0.01923200022429228,
+        "min_ms": 0.018624000251293182,
+        "std_ms": 0.0034253398003137136
+      },
+      "max_time": 0.0011505279541015624,
+      "mean_time": 0.001141315186023712,
+      "memory_allocated_mb": 34.0205078125,
+      "memory_reserved_mb": 1480.0,
+      "min_time": 0.0011371200084686279,
+      "std_time": 4.555471006662272e-06,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 897210.527415803
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s4k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 4,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 32,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.021727999672293663,
+        "mean_ms": 0.017212800029665232,
+        "median_ms": 0.0161920003592968,
+        "min_ms": 0.015231999568641186,
+        "std_ms": 0.002282587014061408
+      },
+      "max_time": 0.0003445119857788086,
+      "mean_time": 0.0003361823946237564,
+      "memory_allocated_mb": 17.015625,
+      "memory_reserved_mb": 1480.0,
+      "min_time": 0.00032927998900413513,
+      "std_time": 4.115851563011574e-06,
+      "tensor_parallel_size": 1,
+      "throughput_tokens_per_sec": 1522982.7860944727
+    }
+  ],
+  "schema_version": "qwen30_vllm020_flashattn_raw.v1"
+}

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/artifacts.sha256

@@ -0,0 +1,8 @@
+e48aa6e6d039bdb68759701c9976f78ad818d6778b6c40b14823a0a4e852c52d  runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/raw/flashattn-code-longctx-tp2.json
+4e6d6002db037bdc71d40f76d6de979ef47960612c15dd985c04048c25bd1d41  runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/aituner.commit
+420e4a613b1bcdb44b873eaa911f4a23129713ff9d486fc783b31a009a55cc85  runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/batch-specs.txt
+5dd612a7806900e9d12afc27ccef196cc5518aea831aa4ed3b0a53bcfb5746cf  runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/max-model-len.txt
+484da176e7355ab5532d5129a8e78bea0ae84b3aa0a42c7f0bea9689667762ff  runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/pip-freeze.txt
+595b25965f76bf7ac25aa3b60126a437a083619303303aaf63346e993862253a  runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/source.sha256
+4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865  runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/vllm-allow-long-max-model-len.txt
+11e8f5af440e3db4c01b519a7b0e30bbcecc3a57bfb53dd86d32e601beb95a32  runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/vllm-source.commit

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/aituner.commit

@@ -0,0 +1 @@
+e251046c306a8e03214fe1b2f40caabb98f76b7b

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/batch-specs.txt

@@ -0,0 +1,13 @@
+q8ks48k
+q8ks64k
+q8ks80k
+q8ks96k
+q8ks112k
+q8ks128k
+q8ks136k
+q512s128k
+q2ks66k
+q4ks100k
+q6ks134k
+q1ks8k
+q512s4k

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/max-model-len.txt

@@ -0,0 +1 @@
+147456

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/pip-freeze.txt

@@ -0,0 +1,179 @@
+aiohappyeyeballs==2.7.1
+aiohttp==3.14.1
+aiosignal==1.4.0
+annotated-doc==0.0.4
+annotated-types==0.7.0
+anthropic==0.117.0
+anyio==4.14.2
+apache-tvm-ffi==0.1.9
+astor==0.8.1
+attrs==26.1.0
+blake3==1.0.9
+cachetools==7.1.4
+cbor2==6.1.3
+certifi==2026.6.17
+cffi==2.1.0
+charset-normalizer==3.4.9
+click==8.4.2
+cloudpickle==3.1.2
+compressed-tensors==0.15.0.1
+cryptography==49.0.0
+cuda-bindings==12.9.7
+cuda-pathfinder==1.5.6
+cuda-python==12.9.7
+cuda-tile==1.5.0
+cuda-toolkit==12.9.1
+depyf==0.20.0
+detect-installer==0.1.0
+dill==0.4.1
+diskcache==5.6.3
+distro==1.9.0
+dnspython==2.8.0
+docstring-parser==0.18.0
+einops==0.8.2
+email-validator==2.3.0
+fastapi==0.139.2
+fastapi-cli==0.0.32
+fastapi-cloud-cli==0.22.2
+fastar==0.11.0
+fastsafetensors==0.3.3
+filelock==3.31.1
+flashinfer-cubin==0.6.8.post1
+flashinfer-python==0.6.8.post1
+frozenlist==1.8.0
+fsspec==2026.6.0
+gguf==0.19.0
+googleapis-common-protos==1.75.0
+grpcio==1.82.1
+h11==0.16.0
+hf-xet==1.5.2
+httpcore==1.0.9
+httptools==0.8.0
+httpx==0.28.1
+httpx-sse==0.4.3
+huggingface-hub==1.24.0
+idna==3.18
+ijson==3.5.1
+interegular==0.3.3
+jinja2==3.1.6
+jiter==0.16.0
+jmespath==1.1.0
+jsonschema==4.26.0
+jsonschema-specifications==2025.9.1
+lark==1.2.2
+llguidance==1.3.0
+llvmlite==0.47.0
+lm-format-enforcer==0.11.3
+loguru==0.7.3
+markdown-it-py==4.2.0
+markupsafe==3.0.3
+mcp==1.28.1
+mdurl==0.1.2
+mistral-common==1.11.6
+ml-dtypes==0.5.4
+model-hosting-container-standards==0.1.16
+mpmath==1.3.0
+msgspec==0.21.1
+multidict==6.7.1
+networkx==3.6.1
+ninja==1.13.0
+numba==0.65.0
+numpy==2.3.5
+nvidia-cublas-cu12==12.9.1.4
+nvidia-cuda-cupti-cu12==12.9.79
+nvidia-cuda-nvrtc-cu12==12.9.86
+nvidia-cuda-runtime-cu12==12.9.79
+nvidia-cudnn-cu12==9.17.1.4
+nvidia-cudnn-frontend==1.18.0
+nvidia-cufft-cu12==11.4.1.4
+nvidia-cufile-cu12==1.14.1.1
+nvidia-curand-cu12==10.3.10.19
+nvidia-cusolver-cu12==11.7.5.82
+nvidia-cusparse-cu12==12.5.10.65
+nvidia-cusparselt-cu12==0.7.1
+nvidia-cutlass-dsl==4.5.3
+nvidia-cutlass-dsl-libs-base==4.5.3
+nvidia-ml-py==13.610.43
+nvidia-nccl-cu12==2.28.9
+nvidia-nvjitlink-cu12==12.9.86
+nvidia-nvshmem-cu12==3.4.5
+nvidia-nvtx-cu12==12.9.79
+openai==2.46.0
+openai-harmony==0.0.8
+opencv-python-headless==5.0.0.93
+opentelemetry-api==1.44.0
+opentelemetry-exporter-otlp==1.44.0
+opentelemetry-exporter-otlp-proto-common==1.44.0
+opentelemetry-exporter-otlp-proto-grpc==1.44.0
+opentelemetry-exporter-otlp-proto-http==1.44.0
+opentelemetry-proto==1.44.0
+opentelemetry-sdk==1.44.0
+opentelemetry-semantic-conventions==0.65b0
+opentelemetry-semantic-conventions-ai==0.5.1
+outlines-core==0.2.14
+packaging==26.2
+partial-json-parser==0.2.1.1.post7
+pillow==12.3.0
+prometheus-client==0.25.0
+prometheus-fastapi-instrumentator==8.0.2
+propcache==0.5.2
+protobuf==7.35.1
+psutil==7.2.2
+py-cpuinfo==9.0.0
+pybase64==1.4.3
+pycountry==26.2.16
+pycparser==3.0
+pydantic==2.13.4
+pydantic-core==2.46.4
+pydantic-extra-types==2.11.1
+pydantic-settings==2.14.2
+pygments==2.20.0
+pyjwt==2.13.0
+python-dotenv==1.2.2
+python-json-logger==4.1.0
+python-multipart==0.0.32
+pyyaml==6.0.3
+pyzmq==27.1.0
+quack-kernels==0.5.0
+referencing==0.37.0
+regex==2026.7.19
+requests==2.34.2
+rich==15.0.0
+rich-toolkit==0.20.3
+rignore==0.8.0
+rpds-py==2026.6.3
+safetensors==0.8.0
+sentencepiece==0.2.2
+sentry-sdk==2.66.0
+setproctitle==1.3.7
+setuptools==80.10.2
+shellingham==1.5.4
+six==1.17.0
+sniffio==1.3.1
+sse-starlette==3.4.5
+starlette==1.3.1
+supervisor==4.3.0
+sympy==1.14.0
+tabulate==0.10.0
+tiktoken==0.13.0
+tilelang==0.1.9
+tokenizers==0.22.2
+torch==2.11.0+cu129
+torch-c-dlpack-ext==0.1.5
+torchaudio==2.11.0+cu129
+torchvision==0.26.0+cu129
+tqdm==4.69.0
+transformers==5.14.1
+triton==3.6.0
+typer==0.27.0
+typing-extensions==4.16.0
+typing-inspection==0.4.2
+urllib3==2.7.0
+uvicorn==0.51.0
+uvloop==0.22.1
+vllm @ https://github.com/vllm-project/vllm/releases/download/v0.20.0/vllm-0.20.0%2Bcu129-cp38-abi3-manylinux_2_31_x86_64.whl
+watchfiles==1.2.0
+websockets==16.1.1
+xgrammar==0.2.3
+yarl==1.24.5
+z3-solver==4.15.4.0

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/source.sha256

@@ -0,0 +1,2 @@
+42381f4bd1b67710d4068eb8993240930326a804c6e5167b0b78df3420e47c74  /home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/../frontier-qwen30-vllm020-profile-v1/profile_vllm020_flashattn.py
+8a7a72b3adb25db6a61a86f9b04d2d19ebbfe3f2d20480852ef2f45376ac3b85  runs/frontier-code-trace-v0/run_flashattn_code_longctx.sh

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/vllm-allow-long-max-model-len.txt

@@ -0,0 +1 @@
+1

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/provenance/vllm-source.commit

@@ -0,0 +1 @@
+88d34c6409e9fb3c7b8ca0c04756f061d2099eb1

runs/frontier-code-trace-v0/profiles/raw/tp2-r1-v2/raw/flashattn-code-longctx-tp2.json

@@ -0,0 +1,564 @@
+{
+  "environment": {
+    "attention_backend": "FLASH_ATTN",
+    "block_size": 16,
+    "dtype": "bfloat16",
+    "gpu": "NVIDIA H20",
+    "model": "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B",
+    "profile_kv_update": true,
+    "profile_method": "cuda_event",
+    "torch_cuda": "12.9",
+    "torch_version": "2.11.0+cu129",
+    "vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1",
+    "vllm_version": "0.20.0"
+  },
+  "rows": [
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks48k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.05100800096988678,
+        "mean_ms": 0.04406719990074635,
+        "median_ms": 0.042847998440265656,
+        "min_ms": 0.04217600077390671,
+        "std_ms": 0.002656672983576728
+      },
+      "max_time": 0.022986656188964845,
+      "mean_time": 0.02263068161010742,
+      "memory_allocated_mb": 120.0849609375,
+      "memory_reserved_mb": 134.0,
+      "min_time": 0.022516607284545898,
+      "std_time": 0.00013840494695459716,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 361986.4457083454
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks64k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.047200001776218414,
+        "mean_ms": 0.043644800409674646,
+        "median_ms": 0.04262400045990944,
+        "min_ms": 0.04182400181889534,
+        "std_ms": 0.001927741871473188
+      },
+      "max_time": 0.030787456512451173,
+      "mean_time": 0.03074285774230957,
+      "memory_allocated_mb": 136.0888671875,
+      "memory_reserved_mb": 198.0,
+      "min_time": 0.030725727081298827,
+      "std_time": 1.9110324860515016e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 266468.39629114367
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks80k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.05049600079655647,
+        "mean_ms": 0.043852799385786054,
+        "median_ms": 0.04289599880576134,
+        "min_ms": 0.04185599833726883,
+        "std_ms": 0.0024573088797598033
+      },
+      "max_time": 0.03898448181152344,
+      "mean_time": 0.038945382690429686,
+      "memory_allocated_mb": 152.0927734375,
+      "memory_reserved_mb": 278.0,
+      "min_time": 0.038930206298828124,
+      "std_time": 1.7734743323340796e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 210345.85961362437
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks96k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.046911999583244324,
+        "mean_ms": 0.04412479996681214,
+        "median_ms": 0.04327999986708164,
+        "min_ms": 0.04227200150489807,
+        "std_ms": 0.001811858548765738
+      },
+      "max_time": 0.04724588775634766,
+      "mean_time": 0.04714819869995117,
+      "memory_allocated_mb": 168.0966796875,
+      "memory_reserved_mb": 374.0,
+      "min_time": 0.047131519317626956,
+      "std_time": 3.329779976561307e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 173750.01009335453
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks112k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.06195199862122536,
+        "mean_ms": 0.04529919996857643,
+        "median_ms": 0.043136000633239746,
+        "min_ms": 0.04182400181889534,
+        "std_ms": 0.005751677082163394
+      },
+      "max_time": 0.05564672088623047,
+      "mean_time": 0.05542207336425782,
+      "memory_allocated_mb": 184.1005859375,
+      "memory_reserved_mb": 486.0,
+      "min_time": 0.05533849716186524,
+      "std_time": 0.00011010194068839293,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 147811.14279429129
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.054687999188899994,
+        "mean_ms": 0.04418559968471527,
+        "median_ms": 0.042767999693751335,
+        "min_ms": 0.04182400181889534,
+        "std_ms": 0.0036820249064621804
+      },
+      "max_time": 0.06380697631835938,
+      "mean_time": 0.06362084197998047,
+      "memory_allocated_mb": 200.1044921875,
+      "memory_reserved_mb": 614.0,
+      "min_time": 0.06354908752441406,
+      "std_time": 8.872184973119366e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 128762.83533905087
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks136k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.06566400080919266,
+        "mean_ms": 0.05008000023663044,
+        "median_ms": 0.04843199998140335,
+        "min_ms": 0.0461760014295578,
+        "std_ms": 0.0055350567765421
+      },
+      "max_time": 0.06791651153564453,
+      "mean_time": 0.06770619888305665,
+      "memory_allocated_mb": 208.1064453125,
+      "memory_reserved_mb": 750.0,
+      "min_time": 0.06764147186279297,
+      "std_time": 9.93520482760623e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 120993.35267881997
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03859199956059456,
+        "mean_ms": 0.02689919974654913,
+        "median_ms": 0.0244159996509552,
+        "min_ms": 0.023711999878287315,
+        "std_ms": 0.004626699769228834
+      },
+      "max_time": 0.004829855918884277,
+      "mean_time": 0.004790303993225097,
+      "memory_allocated_mb": 132.5458984375,
+      "memory_reserved_mb": 750.0,
+      "min_time": 0.004775519847869873,
+      "std_time": 1.597276061319242e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 106882.56960813323
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q2ks66k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.0453759990632534,
+        "mean_ms": 0.027676799893379213,
+        "median_ms": 0.025200000032782555,
+        "min_ms": 0.024224000051617622,
+        "std_ms": 0.006154880946840619
+      },
+      "max_time": 0.00963871955871582,
+      "mean_time": 0.00959802885055542,
+      "memory_allocated_mb": 84.04248046875,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.009579392433166503,
+      "std_time": 1.6811667352984867e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 213377.14565022237
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q4ks100k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03593600168824196,
+        "mean_ms": 0.02780479993671179,
+        "median_ms": 0.027040000073611736,
+        "min_ms": 0.02630399912595749,
+        "std_ms": 0.0027569987978884785
+      },
+      "max_time": 0.025383039474487303,
+      "mean_time": 0.025250255966186526,
+      "memory_allocated_mb": 136.06640625,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.025216863632202147,
+      "std_time": 4.761963064809279e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 162216.17735222535
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q6ks134k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.041728001087903976,
+        "mean_ms": 0.03665280006825924,
+        "median_ms": 0.0352960005402565,
+        "min_ms": 0.034015998244285583,
+        "std_ms": 0.002710617869728372
+      },
+      "max_time": 0.048498847961425784,
+      "mean_time": 0.048172192001342776,
+      "memory_allocated_mb": 188.09033203125,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.04810128021240234,
+      "std_time": 0.00011195495368044632,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 127542.46266868527
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q1ks8k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.02191999927163124,
+        "mean_ms": 0.016563199926167727,
+        "median_ms": 0.015584000386297703,
+        "min_ms": 0.014879999682307243,
+        "std_ms": 0.002306362959594964
+      },
+      "max_time": 0.0006153280138969421,
+      "mean_time": 0.0006064576089382171,
+      "memory_allocated_mb": 17.0205078125,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.0006022400259971619,
+      "std_time": 4.680714919723724e-06,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 1688493.9440248988
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s4k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03843199834227562,
+        "mean_ms": 0.019180799927562477,
+        "median_ms": 0.01601599995046854,
+        "min_ms": 0.014911999925971031,
+        "std_ms": 0.006856655009795906
+      },
+      "max_time": 0.00022172799706459046,
+      "mean_time": 0.00020788159817457198,
+      "memory_allocated_mb": 8.515625,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.00019884799420833587,
+      "std_time": 7.012280869814824e-06,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 2462940.464648726
+    }
+  ],
+  "schema_version": "qwen30_vllm020_flashattn_raw.v1"
+}

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/artifacts.sha256

@@ -0,0 +1,8 @@
+e89bb293d5d8d6d4ec226de86ce6e4cf46daaf6eefb5b3f2809a7d079b948ecf  runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/raw/flashattn-code-longctx-tp2.json
+4e6d6002db037bdc71d40f76d6de979ef47960612c15dd985c04048c25bd1d41  runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/aituner.commit
+420e4a613b1bcdb44b873eaa911f4a23129713ff9d486fc783b31a009a55cc85  runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/batch-specs.txt
+5dd612a7806900e9d12afc27ccef196cc5518aea831aa4ed3b0a53bcfb5746cf  runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/max-model-len.txt
+484da176e7355ab5532d5129a8e78bea0ae84b3aa0a42c7f0bea9689667762ff  runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/pip-freeze.txt
+595b25965f76bf7ac25aa3b60126a437a083619303303aaf63346e993862253a  runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/source.sha256
+4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865  runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/vllm-allow-long-max-model-len.txt
+11e8f5af440e3db4c01b519a7b0e30bbcecc3a57bfb53dd86d32e601beb95a32  runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/vllm-source.commit

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/aituner.commit

@@ -0,0 +1 @@
+e251046c306a8e03214fe1b2f40caabb98f76b7b

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/batch-specs.txt

@@ -0,0 +1,13 @@
+q8ks48k
+q8ks64k
+q8ks80k
+q8ks96k
+q8ks112k
+q8ks128k
+q8ks136k
+q512s128k
+q2ks66k
+q4ks100k
+q6ks134k
+q1ks8k
+q512s4k

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/max-model-len.txt

@@ -0,0 +1 @@
+147456

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/pip-freeze.txt

@@ -0,0 +1,179 @@
+aiohappyeyeballs==2.7.1
+aiohttp==3.14.1
+aiosignal==1.4.0
+annotated-doc==0.0.4
+annotated-types==0.7.0
+anthropic==0.117.0
+anyio==4.14.2
+apache-tvm-ffi==0.1.9
+astor==0.8.1
+attrs==26.1.0
+blake3==1.0.9
+cachetools==7.1.4
+cbor2==6.1.3
+certifi==2026.6.17
+cffi==2.1.0
+charset-normalizer==3.4.9
+click==8.4.2
+cloudpickle==3.1.2
+compressed-tensors==0.15.0.1
+cryptography==49.0.0
+cuda-bindings==12.9.7
+cuda-pathfinder==1.5.6
+cuda-python==12.9.7
+cuda-tile==1.5.0
+cuda-toolkit==12.9.1
+depyf==0.20.0
+detect-installer==0.1.0
+dill==0.4.1
+diskcache==5.6.3
+distro==1.9.0
+dnspython==2.8.0
+docstring-parser==0.18.0
+einops==0.8.2
+email-validator==2.3.0
+fastapi==0.139.2
+fastapi-cli==0.0.32
+fastapi-cloud-cli==0.22.2
+fastar==0.11.0
+fastsafetensors==0.3.3
+filelock==3.31.1
+flashinfer-cubin==0.6.8.post1
+flashinfer-python==0.6.8.post1
+frozenlist==1.8.0
+fsspec==2026.6.0
+gguf==0.19.0
+googleapis-common-protos==1.75.0
+grpcio==1.82.1
+h11==0.16.0
+hf-xet==1.5.2
+httpcore==1.0.9
+httptools==0.8.0
+httpx==0.28.1
+httpx-sse==0.4.3
+huggingface-hub==1.24.0
+idna==3.18
+ijson==3.5.1
+interegular==0.3.3
+jinja2==3.1.6
+jiter==0.16.0
+jmespath==1.1.0
+jsonschema==4.26.0
+jsonschema-specifications==2025.9.1
+lark==1.2.2
+llguidance==1.3.0
+llvmlite==0.47.0
+lm-format-enforcer==0.11.3
+loguru==0.7.3
+markdown-it-py==4.2.0
+markupsafe==3.0.3
+mcp==1.28.1
+mdurl==0.1.2
+mistral-common==1.11.6
+ml-dtypes==0.5.4
+model-hosting-container-standards==0.1.16
+mpmath==1.3.0
+msgspec==0.21.1
+multidict==6.7.1
+networkx==3.6.1
+ninja==1.13.0
+numba==0.65.0
+numpy==2.3.5
+nvidia-cublas-cu12==12.9.1.4
+nvidia-cuda-cupti-cu12==12.9.79
+nvidia-cuda-nvrtc-cu12==12.9.86
+nvidia-cuda-runtime-cu12==12.9.79
+nvidia-cudnn-cu12==9.17.1.4
+nvidia-cudnn-frontend==1.18.0
+nvidia-cufft-cu12==11.4.1.4
+nvidia-cufile-cu12==1.14.1.1
+nvidia-curand-cu12==10.3.10.19
+nvidia-cusolver-cu12==11.7.5.82
+nvidia-cusparse-cu12==12.5.10.65
+nvidia-cusparselt-cu12==0.7.1
+nvidia-cutlass-dsl==4.5.3
+nvidia-cutlass-dsl-libs-base==4.5.3
+nvidia-ml-py==13.610.43
+nvidia-nccl-cu12==2.28.9
+nvidia-nvjitlink-cu12==12.9.86
+nvidia-nvshmem-cu12==3.4.5
+nvidia-nvtx-cu12==12.9.79
+openai==2.46.0
+openai-harmony==0.0.8
+opencv-python-headless==5.0.0.93
+opentelemetry-api==1.44.0
+opentelemetry-exporter-otlp==1.44.0
+opentelemetry-exporter-otlp-proto-common==1.44.0
+opentelemetry-exporter-otlp-proto-grpc==1.44.0
+opentelemetry-exporter-otlp-proto-http==1.44.0
+opentelemetry-proto==1.44.0
+opentelemetry-sdk==1.44.0
+opentelemetry-semantic-conventions==0.65b0
+opentelemetry-semantic-conventions-ai==0.5.1
+outlines-core==0.2.14
+packaging==26.2
+partial-json-parser==0.2.1.1.post7
+pillow==12.3.0
+prometheus-client==0.25.0
+prometheus-fastapi-instrumentator==8.0.2
+propcache==0.5.2
+protobuf==7.35.1
+psutil==7.2.2
+py-cpuinfo==9.0.0
+pybase64==1.4.3
+pycountry==26.2.16
+pycparser==3.0
+pydantic==2.13.4
+pydantic-core==2.46.4
+pydantic-extra-types==2.11.1
+pydantic-settings==2.14.2
+pygments==2.20.0
+pyjwt==2.13.0
+python-dotenv==1.2.2
+python-json-logger==4.1.0
+python-multipart==0.0.32
+pyyaml==6.0.3
+pyzmq==27.1.0
+quack-kernels==0.5.0
+referencing==0.37.0
+regex==2026.7.19
+requests==2.34.2
+rich==15.0.0
+rich-toolkit==0.20.3
+rignore==0.8.0
+rpds-py==2026.6.3
+safetensors==0.8.0
+sentencepiece==0.2.2
+sentry-sdk==2.66.0
+setproctitle==1.3.7
+setuptools==80.10.2
+shellingham==1.5.4
+six==1.17.0
+sniffio==1.3.1
+sse-starlette==3.4.5
+starlette==1.3.1
+supervisor==4.3.0
+sympy==1.14.0
+tabulate==0.10.0
+tiktoken==0.13.0
+tilelang==0.1.9
+tokenizers==0.22.2
+torch==2.11.0+cu129
+torch-c-dlpack-ext==0.1.5
+torchaudio==2.11.0+cu129
+torchvision==0.26.0+cu129
+tqdm==4.69.0
+transformers==5.14.1
+triton==3.6.0
+typer==0.27.0
+typing-extensions==4.16.0
+typing-inspection==0.4.2
+urllib3==2.7.0
+uvicorn==0.51.0
+uvloop==0.22.1
+vllm @ https://github.com/vllm-project/vllm/releases/download/v0.20.0/vllm-0.20.0%2Bcu129-cp38-abi3-manylinux_2_31_x86_64.whl
+watchfiles==1.2.0
+websockets==16.1.1
+xgrammar==0.2.3
+yarl==1.24.5
+z3-solver==4.15.4.0

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/source.sha256

@@ -0,0 +1,2 @@
+42381f4bd1b67710d4068eb8993240930326a804c6e5167b0b78df3420e47c74  /home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/../frontier-qwen30-vllm020-profile-v1/profile_vllm020_flashattn.py
+8a7a72b3adb25db6a61a86f9b04d2d19ebbfe3f2d20480852ef2f45376ac3b85  runs/frontier-code-trace-v0/run_flashattn_code_longctx.sh

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/vllm-allow-long-max-model-len.txt

@@ -0,0 +1 @@
+1

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/provenance/vllm-source.commit

@@ -0,0 +1 @@
+88d34c6409e9fb3c7b8ca0c04756f061d2099eb1

runs/frontier-code-trace-v0/profiles/raw/tp2-r2-v2/raw/flashattn-code-longctx-tp2.json

@@ -0,0 +1,564 @@
+{
+  "environment": {
+    "attention_backend": "FLASH_ATTN",
+    "block_size": 16,
+    "dtype": "bfloat16",
+    "gpu": "NVIDIA H20",
+    "model": "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B",
+    "profile_kv_update": true,
+    "profile_method": "cuda_event",
+    "torch_cuda": "12.9",
+    "torch_version": "2.11.0+cu129",
+    "vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1",
+    "vllm_version": "0.20.0"
+  },
+  "rows": [
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks48k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.047520000487565994,
+        "mean_ms": 0.04390720054507256,
+        "median_ms": 0.043088000267744064,
+        "min_ms": 0.041760001331567764,
+        "std_ms": 0.0020753894539536763
+      },
+      "max_time": 0.0226856632232666,
+      "mean_time": 0.022550320053100585,
+      "memory_allocated_mb": 120.0849609375,
+      "memory_reserved_mb": 134.0,
+      "min_time": 0.02251683235168457,
+      "std_time": 5.03848780842368e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 363276.440454495
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks64k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.05392000079154968,
+        "mean_ms": 0.04477120004594326,
+        "median_ms": 0.04323200136423111,
+        "min_ms": 0.04150399938225746,
+        "std_ms": 0.003602053060896233
+      },
+      "max_time": 0.03088857650756836,
+      "mean_time": 0.030766559982299803,
+      "memory_allocated_mb": 136.0888671875,
+      "memory_reserved_mb": 198.0,
+      "min_time": 0.030721696853637695,
+      "std_time": 5.739627345668801e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 266263.11179127306
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks80k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.04835199937224388,
+        "mean_ms": 0.04382400028407574,
+        "median_ms": 0.0427200011909008,
+        "min_ms": 0.041600000113248825,
+        "std_ms": 0.002316465883417635
+      },
+      "max_time": 0.03911993789672852,
+      "mean_time": 0.03901147232055664,
+      "memory_allocated_mb": 152.0927734375,
+      "memory_reserved_mb": 278.0,
+      "min_time": 0.03894742584228516,
+      "std_time": 5.231496369987919e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 209989.51110295113
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks96k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.04902400076389313,
+        "mean_ms": 0.04363839998841286,
+        "median_ms": 0.04273599945008755,
+        "min_ms": 0.04153599962592125,
+        "std_ms": 0.002234876899495284
+      },
+      "max_time": 0.047580448150634766,
+      "mean_time": 0.0472187198638916,
+      "memory_allocated_mb": 168.0966796875,
+      "memory_reserved_mb": 374.0,
+      "min_time": 0.04714566421508789,
+      "std_time": 0.00012505866815108255,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 173490.51443185066
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks112k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.04848000034689903,
+        "mean_ms": 0.04378239996731281,
+        "median_ms": 0.042527999728918076,
+        "min_ms": 0.04163200035691261,
+        "std_ms": 0.002414242667964844
+      },
+      "max_time": 0.05583651351928711,
+      "mean_time": 0.055455712127685554,
+      "memory_allocated_mb": 184.1005859375,
+      "memory_reserved_mb": 486.0,
+      "min_time": 0.05535052871704101,
+      "std_time": 0.00014272069779462445,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 147721.48234501254
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.04851200059056282,
+        "mean_ms": 0.04361280016601086,
+        "median_ms": 0.042128000408411026,
+        "min_ms": 0.04179200157523155,
+        "std_ms": 0.0023557503035821635
+      },
+      "max_time": 0.06379235076904297,
+      "mean_time": 0.06362125053405762,
+      "memory_allocated_mb": 200.1044921875,
+      "memory_reserved_mb": 614.0,
+      "min_time": 0.06355436706542969,
+      "std_time": 7.081883106781061e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 128762.00846782589
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks136k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.05718399956822395,
+        "mean_ms": 0.04485439993441105,
+        "median_ms": 0.042527999728918076,
+        "min_ms": 0.04156799986958504,
+        "std_ms": 0.004775190073319638
+      },
+      "max_time": 0.06783235168457032,
+      "mean_time": 0.06771669235229492,
+      "memory_allocated_mb": 208.1064453125,
+      "memory_reserved_mb": 750.0,
+      "min_time": 0.06765017700195312,
+      "std_time": 5.8650788028093655e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 120974.603387024
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.019807999953627586,
+        "mean_ms": 0.015823999978601934,
+        "median_ms": 0.015039999969303608,
+        "min_ms": 0.014303999952971935,
+        "std_ms": 0.001913656346272671
+      },
+      "max_time": 0.004766176223754883,
+      "mean_time": 0.0047575551986694335,
+      "memory_allocated_mb": 132.5458984375,
+      "memory_reserved_mb": 750.0,
+      "min_time": 0.004751776218414307,
+      "std_time": 4.190866412679079e-06,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 107618.29944573072
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q2ks66k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.02687999978661537,
+        "mean_ms": 0.01938559990376234,
+        "median_ms": 0.018383999355137348,
+        "min_ms": 0.018015999346971512,
+        "std_ms": 0.0025744710494052777
+      },
+      "max_time": 0.009582624435424805,
+      "mean_time": 0.009561939239501951,
+      "memory_allocated_mb": 84.04248046875,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.009554431915283204,
+      "std_time": 8.291579854680458e-06,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 214182.49464913702
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q4ks100k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03206399828195572,
+        "mean_ms": 0.028105599991977214,
+        "median_ms": 0.02675200067460537,
+        "min_ms": 0.025728000327944756,
+        "std_ms": 0.0022390414558466644
+      },
+      "max_time": 0.02538035202026367,
+      "mean_time": 0.0252856897354126,
+      "memory_allocated_mb": 136.06640625,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.025221471786499024,
+      "std_time": 4.676405035080222e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 161988.85784252718
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q6ks134k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.04089599847793579,
+        "mean_ms": 0.03524160049855709,
+        "median_ms": 0.03411199897527695,
+        "min_ms": 0.03385600075125694,
+        "std_ms": 0.0022532652248236193
+      },
+      "max_time": 0.048259136199951175,
+      "mean_time": 0.048143408584594725,
+      "memory_allocated_mb": 188.09033203125,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.048105857849121095,
+      "std_time": 5.1879732806325464e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 127618.7162611083
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q1ks8k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.018783999606966972,
+        "mean_ms": 0.015311999991536141,
+        "median_ms": 0.014943999703973532,
+        "min_ms": 0.014271999709308147,
+        "std_ms": 0.0012243293501672883
+      },
+      "max_time": 0.0008861759901046753,
+      "mean_time": 0.0006353183984756471,
+      "memory_allocated_mb": 17.0205078125,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.0006019840240478516,
+      "std_time": 8.386037831062328e-05,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 1611790.2495141604
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s4k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 2,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 16,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.019840000197291374,
+        "mean_ms": 0.015875199995934963,
+        "median_ms": 0.01497599994763732,
+        "min_ms": 0.014431999996304512,
+        "std_ms": 0.001896142444948042
+      },
+      "max_time": 0.00022230400145053863,
+      "mean_time": 0.00020609600096940998,
+      "memory_allocated_mb": 8.515625,
+      "memory_reserved_mb": 752.0,
+      "min_time": 0.00019760000705718993,
+      "std_time": 7.520956759601396e-06,
+      "tensor_parallel_size": 2,
+      "throughput_tokens_per_sec": 2484279.1591865686
+    }
+  ],
+  "schema_version": "qwen30_vllm020_flashattn_raw.v1"
+}

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/artifacts.sha256

@@ -0,0 +1,8 @@
+759d33decc09d229d07818df8161ed41180cab575304f335ad4d5e7a917cbc93  runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/raw/flashattn-code-longctx-tp4.json
+4e6d6002db037bdc71d40f76d6de979ef47960612c15dd985c04048c25bd1d41  runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/aituner.commit
+420e4a613b1bcdb44b873eaa911f4a23129713ff9d486fc783b31a009a55cc85  runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/batch-specs.txt
+5dd612a7806900e9d12afc27ccef196cc5518aea831aa4ed3b0a53bcfb5746cf  runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/max-model-len.txt
+484da176e7355ab5532d5129a8e78bea0ae84b3aa0a42c7f0bea9689667762ff  runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/pip-freeze.txt
+595b25965f76bf7ac25aa3b60126a437a083619303303aaf63346e993862253a  runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/source.sha256
+4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865  runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/vllm-allow-long-max-model-len.txt
+11e8f5af440e3db4c01b519a7b0e30bbcecc3a57bfb53dd86d32e601beb95a32  runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/vllm-source.commit

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/aituner.commit

@@ -0,0 +1 @@
+e251046c306a8e03214fe1b2f40caabb98f76b7b

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/batch-specs.txt

@@ -0,0 +1,13 @@
+q8ks48k
+q8ks64k
+q8ks80k
+q8ks96k
+q8ks112k
+q8ks128k
+q8ks136k
+q512s128k
+q2ks66k
+q4ks100k
+q6ks134k
+q1ks8k
+q512s4k

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/max-model-len.txt

@@ -0,0 +1 @@
+147456

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/pip-freeze.txt

@@ -0,0 +1,179 @@
+aiohappyeyeballs==2.7.1
+aiohttp==3.14.1
+aiosignal==1.4.0
+annotated-doc==0.0.4
+annotated-types==0.7.0
+anthropic==0.117.0
+anyio==4.14.2
+apache-tvm-ffi==0.1.9
+astor==0.8.1
+attrs==26.1.0
+blake3==1.0.9
+cachetools==7.1.4
+cbor2==6.1.3
+certifi==2026.6.17
+cffi==2.1.0
+charset-normalizer==3.4.9
+click==8.4.2
+cloudpickle==3.1.2
+compressed-tensors==0.15.0.1
+cryptography==49.0.0
+cuda-bindings==12.9.7
+cuda-pathfinder==1.5.6
+cuda-python==12.9.7
+cuda-tile==1.5.0
+cuda-toolkit==12.9.1
+depyf==0.20.0
+detect-installer==0.1.0
+dill==0.4.1
+diskcache==5.6.3
+distro==1.9.0
+dnspython==2.8.0
+docstring-parser==0.18.0
+einops==0.8.2
+email-validator==2.3.0
+fastapi==0.139.2
+fastapi-cli==0.0.32
+fastapi-cloud-cli==0.22.2
+fastar==0.11.0
+fastsafetensors==0.3.3
+filelock==3.31.1
+flashinfer-cubin==0.6.8.post1
+flashinfer-python==0.6.8.post1
+frozenlist==1.8.0
+fsspec==2026.6.0
+gguf==0.19.0
+googleapis-common-protos==1.75.0
+grpcio==1.82.1
+h11==0.16.0
+hf-xet==1.5.2
+httpcore==1.0.9
+httptools==0.8.0
+httpx==0.28.1
+httpx-sse==0.4.3
+huggingface-hub==1.24.0
+idna==3.18
+ijson==3.5.1
+interegular==0.3.3
+jinja2==3.1.6
+jiter==0.16.0
+jmespath==1.1.0
+jsonschema==4.26.0
+jsonschema-specifications==2025.9.1
+lark==1.2.2
+llguidance==1.3.0
+llvmlite==0.47.0
+lm-format-enforcer==0.11.3
+loguru==0.7.3
+markdown-it-py==4.2.0
+markupsafe==3.0.3
+mcp==1.28.1
+mdurl==0.1.2
+mistral-common==1.11.6
+ml-dtypes==0.5.4
+model-hosting-container-standards==0.1.16
+mpmath==1.3.0
+msgspec==0.21.1
+multidict==6.7.1
+networkx==3.6.1
+ninja==1.13.0
+numba==0.65.0
+numpy==2.3.5
+nvidia-cublas-cu12==12.9.1.4
+nvidia-cuda-cupti-cu12==12.9.79
+nvidia-cuda-nvrtc-cu12==12.9.86
+nvidia-cuda-runtime-cu12==12.9.79
+nvidia-cudnn-cu12==9.17.1.4
+nvidia-cudnn-frontend==1.18.0
+nvidia-cufft-cu12==11.4.1.4
+nvidia-cufile-cu12==1.14.1.1
+nvidia-curand-cu12==10.3.10.19
+nvidia-cusolver-cu12==11.7.5.82
+nvidia-cusparse-cu12==12.5.10.65
+nvidia-cusparselt-cu12==0.7.1
+nvidia-cutlass-dsl==4.5.3
+nvidia-cutlass-dsl-libs-base==4.5.3
+nvidia-ml-py==13.610.43
+nvidia-nccl-cu12==2.28.9
+nvidia-nvjitlink-cu12==12.9.86
+nvidia-nvshmem-cu12==3.4.5
+nvidia-nvtx-cu12==12.9.79
+openai==2.46.0
+openai-harmony==0.0.8
+opencv-python-headless==5.0.0.93
+opentelemetry-api==1.44.0
+opentelemetry-exporter-otlp==1.44.0
+opentelemetry-exporter-otlp-proto-common==1.44.0
+opentelemetry-exporter-otlp-proto-grpc==1.44.0
+opentelemetry-exporter-otlp-proto-http==1.44.0
+opentelemetry-proto==1.44.0
+opentelemetry-sdk==1.44.0
+opentelemetry-semantic-conventions==0.65b0
+opentelemetry-semantic-conventions-ai==0.5.1
+outlines-core==0.2.14
+packaging==26.2
+partial-json-parser==0.2.1.1.post7
+pillow==12.3.0
+prometheus-client==0.25.0
+prometheus-fastapi-instrumentator==8.0.2
+propcache==0.5.2
+protobuf==7.35.1
+psutil==7.2.2
+py-cpuinfo==9.0.0
+pybase64==1.4.3
+pycountry==26.2.16
+pycparser==3.0
+pydantic==2.13.4
+pydantic-core==2.46.4
+pydantic-extra-types==2.11.1
+pydantic-settings==2.14.2
+pygments==2.20.0
+pyjwt==2.13.0
+python-dotenv==1.2.2
+python-json-logger==4.1.0
+python-multipart==0.0.32
+pyyaml==6.0.3
+pyzmq==27.1.0
+quack-kernels==0.5.0
+referencing==0.37.0
+regex==2026.7.19
+requests==2.34.2
+rich==15.0.0
+rich-toolkit==0.20.3
+rignore==0.8.0
+rpds-py==2026.6.3
+safetensors==0.8.0
+sentencepiece==0.2.2
+sentry-sdk==2.66.0
+setproctitle==1.3.7
+setuptools==80.10.2
+shellingham==1.5.4
+six==1.17.0
+sniffio==1.3.1
+sse-starlette==3.4.5
+starlette==1.3.1
+supervisor==4.3.0
+sympy==1.14.0
+tabulate==0.10.0
+tiktoken==0.13.0
+tilelang==0.1.9
+tokenizers==0.22.2
+torch==2.11.0+cu129
+torch-c-dlpack-ext==0.1.5
+torchaudio==2.11.0+cu129
+torchvision==0.26.0+cu129
+tqdm==4.69.0
+transformers==5.14.1
+triton==3.6.0
+typer==0.27.0
+typing-extensions==4.16.0
+typing-inspection==0.4.2
+urllib3==2.7.0
+uvicorn==0.51.0
+uvloop==0.22.1
+vllm @ https://github.com/vllm-project/vllm/releases/download/v0.20.0/vllm-0.20.0%2Bcu129-cp38-abi3-manylinux_2_31_x86_64.whl
+watchfiles==1.2.0
+websockets==16.1.1
+xgrammar==0.2.3
+yarl==1.24.5
+z3-solver==4.15.4.0

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/source.sha256

@@ -0,0 +1,2 @@
+42381f4bd1b67710d4068eb8993240930326a804c6e5167b0b78df3420e47c74  /home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/../frontier-qwen30-vllm020-profile-v1/profile_vllm020_flashattn.py
+8a7a72b3adb25db6a61a86f9b04d2d19ebbfe3f2d20480852ef2f45376ac3b85  runs/frontier-code-trace-v0/run_flashattn_code_longctx.sh

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/vllm-allow-long-max-model-len.txt

@@ -0,0 +1 @@
+1

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/provenance/vllm-source.commit

@@ -0,0 +1 @@
+88d34c6409e9fb3c7b8ca0c04756f061d2099eb1

runs/frontier-code-trace-v0/profiles/raw/tp4-r1-v2/raw/flashattn-code-longctx-tp4.json

@@ -0,0 +1,564 @@
+{
+  "environment": {
+    "attention_backend": "FLASH_ATTN",
+    "block_size": 16,
+    "dtype": "bfloat16",
+    "gpu": "NVIDIA H20",
+    "model": "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B",
+    "profile_kv_update": true,
+    "profile_method": "cuda_event",
+    "torch_cuda": "12.9",
+    "torch_version": "2.11.0+cu129",
+    "vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1",
+    "vllm_version": "0.20.0"
+  },
+  "rows": [
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks48k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.039583999663591385,
+        "mean_ms": 0.027286400087177753,
+        "median_ms": 0.025200000032782555,
+        "min_ms": 0.024191999807953835,
+        "std_ms": 0.004620118437533604
+      },
+      "max_time": 0.011496224403381348,
+      "mean_time": 0.011354345512390137,
+      "memory_allocated_mb": 60.0849609375,
+      "memory_reserved_mb": 62.0,
+      "min_time": 0.01132140827178955,
+      "std_time": 5.011776621543997e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 721485.8831854897
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks64k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03232000023126602,
+        "mean_ms": 0.026406400091946124,
+        "median_ms": 0.02518399991095066,
+        "min_ms": 0.024032000452280045,
+        "std_ms": 0.002792696117638337
+      },
+      "max_time": 0.015639776229858397,
+      "mean_time": 0.015451993656158448,
+      "memory_allocated_mb": 68.0888671875,
+      "memory_reserved_mb": 94.0,
+      "min_time": 0.01542249584197998,
+      "std_time": 6.304729308178388e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 530158.1260185833
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks80k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03488000109791756,
+        "mean_ms": 0.026147199980914592,
+        "median_ms": 0.02459200005978346,
+        "min_ms": 0.024224000051617622,
+        "std_ms": 0.003303059079404486
+      },
+      "max_time": 0.01970569610595703,
+      "mean_time": 0.01954985942840576,
+      "memory_allocated_mb": 76.0927734375,
+      "memory_reserved_mb": 134.0,
+      "min_time": 0.019525152206420898,
+      "std_time": 5.224891496533832e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 419031.1459783236
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks96k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03046399913728237,
+        "mean_ms": 0.0255103999748826,
+        "median_ms": 0.024656000547111034,
+        "min_ms": 0.024159999564290047,
+        "std_ms": 0.00195717586616911
+      },
+      "max_time": 0.023927040100097656,
+      "mean_time": 0.02367282257080078,
+      "memory_allocated_mb": 84.0966796875,
+      "memory_reserved_mb": 182.0,
+      "min_time": 0.023630016326904296,
+      "std_time": 8.715594728873713e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 346050.83426360885
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks112k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03574400022625923,
+        "mean_ms": 0.02678720001131296,
+        "median_ms": 0.024720000103116035,
+        "min_ms": 0.02380800060927868,
+        "std_ms": 0.003678231234048846
+      },
+      "max_time": 0.02779840087890625,
+      "mean_time": 0.027747158622741696,
+      "memory_allocated_mb": 92.1005859375,
+      "memory_reserved_mb": 238.0,
+      "min_time": 0.027736671447753908,
+      "std_time": 1.8952179343788658e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 295237.4371509809
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.042080000042915344,
+        "mean_ms": 0.02844479978084564,
+        "median_ms": 0.02527999971061945,
+        "min_ms": 0.024224000051617622,
+        "std_ms": 0.006562862750709003
+      },
+      "max_time": 0.03208835220336914,
+      "mean_time": 0.03186994876861572,
+      "memory_allocated_mb": 100.1044921875,
+      "memory_reserved_mb": 302.0,
+      "min_time": 0.03183427238464356,
+      "std_time": 7.444245241186317e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 257044.65543625728
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks136k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03139200061559677,
+        "mean_ms": 0.026396799832582474,
+        "median_ms": 0.025071999989449978,
+        "min_ms": 0.02377600036561489,
+        "std_ms": 0.0025962810796740263
+      },
+      "max_time": 0.033969856262207034,
+      "mean_time": 0.033897081375122075,
+      "memory_allocated_mb": 104.1064453125,
+      "memory_reserved_mb": 370.0,
+      "min_time": 0.03388313674926758,
+      "std_time": 2.518493598134421e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 241672.72424853416
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.022624000906944275,
+        "mean_ms": 0.017174400109797715,
+        "median_ms": 0.015536000020802021,
+        "min_ms": 0.014944000169634819,
+        "std_ms": 0.0028331860349340154
+      },
+      "max_time": 0.003192960023880005,
+      "mean_time": 0.003182867193222046,
+      "memory_allocated_mb": 66.2958984375,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.003179744005203247,
+      "std_time": 3.5239767128429387e-06,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 160861.25148115202
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q2ks66k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.02985600009560585,
+        "mean_ms": 0.017318400088697672,
+        "median_ms": 0.015568000264465809,
+        "min_ms": 0.014655999839305878,
+        "std_ms": 0.004479309643844885
+      },
+      "max_time": 0.004826848030090332,
+      "mean_time": 0.004817302370071412,
+      "memory_allocated_mb": 42.04248046875,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.004811935901641846,
+      "std_time": 5.94674080057053e-06,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 425134.20222979283
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q4ks100k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03916800022125244,
+        "mean_ms": 0.021356799826025962,
+        "median_ms": 0.01935999933630228,
+        "min_ms": 0.017184000462293625,
+        "std_ms": 0.0062801287963799276
+      },
+      "max_time": 0.014635007858276367,
+      "mean_time": 0.014412275123596191,
+      "memory_allocated_mb": 68.06640625,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.014374591827392579,
+      "std_time": 7.569270440104603e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 284202.17938345566
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q6ks134k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.033215999603271484,
+        "mean_ms": 0.023340800032019614,
+        "median_ms": 0.02092800009995699,
+        "min_ms": 0.020479999482631683,
+        "std_ms": 0.004304974035052475
+      },
+      "max_time": 0.02418492889404297,
+      "mean_time": 0.02411243553161621,
+      "memory_allocated_mb": 96.09033203125,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.024088096618652344,
+      "std_time": 2.797845886790172e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 254806.28001862322
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q1ks8k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.04992000013589859,
+        "mean_ms": 0.019958400167524815,
+        "median_ms": 0.015552000608295202,
+        "min_ms": 0.014688000082969666,
+        "std_ms": 0.010290953685254347
+      },
+      "max_time": 0.00036675199866294863,
+      "mean_time": 0.0003469599992036819,
+      "memory_allocated_mb": 8.5205078125,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.0003364480137825012,
+      "std_time": 7.875187259592915e-06,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 2951348.865431786
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s4k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.022336000576615334,
+        "mean_ms": 0.016252800077199935,
+        "median_ms": 0.014895999804139137,
+        "min_ms": 0.014271999709308147,
+        "std_ms": 0.0028271219653727385
+      },
+      "max_time": 0.00016867199540138245,
+      "mean_time": 0.00015103680044412614,
+      "memory_allocated_mb": 4.265625,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.00014329600334167482,
+      "std_time": 7.872199103975806e-06,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 3389902.3184710997
+    }
+  ],
+  "schema_version": "qwen30_vllm020_flashattn_raw.v1"
+}

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/artifacts.sha256

@@ -0,0 +1,8 @@
+6ee8fdf15bef211e7d53fef0c5daf02b17429bc7d88a49cb967c2b2f6134ebe2  runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/raw/flashattn-code-longctx-tp4.json
+4e6d6002db037bdc71d40f76d6de979ef47960612c15dd985c04048c25bd1d41  runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/aituner.commit
+420e4a613b1bcdb44b873eaa911f4a23129713ff9d486fc783b31a009a55cc85  runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/batch-specs.txt
+5dd612a7806900e9d12afc27ccef196cc5518aea831aa4ed3b0a53bcfb5746cf  runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/max-model-len.txt
+484da176e7355ab5532d5129a8e78bea0ae84b3aa0a42c7f0bea9689667762ff  runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/pip-freeze.txt
+595b25965f76bf7ac25aa3b60126a437a083619303303aaf63346e993862253a  runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/source.sha256
+4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865  runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/vllm-allow-long-max-model-len.txt
+11e8f5af440e3db4c01b519a7b0e30bbcecc3a57bfb53dd86d32e601beb95a32  runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/vllm-source.commit

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/aituner.commit

@@ -0,0 +1 @@
+e251046c306a8e03214fe1b2f40caabb98f76b7b

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/batch-specs.txt

@@ -0,0 +1,13 @@
+q8ks48k
+q8ks64k
+q8ks80k
+q8ks96k
+q8ks112k
+q8ks128k
+q8ks136k
+q512s128k
+q2ks66k
+q4ks100k
+q6ks134k
+q1ks8k
+q512s4k

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/max-model-len.txt

@@ -0,0 +1 @@
+147456

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/pip-freeze.txt

@@ -0,0 +1,179 @@
+aiohappyeyeballs==2.7.1
+aiohttp==3.14.1
+aiosignal==1.4.0
+annotated-doc==0.0.4
+annotated-types==0.7.0
+anthropic==0.117.0
+anyio==4.14.2
+apache-tvm-ffi==0.1.9
+astor==0.8.1
+attrs==26.1.0
+blake3==1.0.9
+cachetools==7.1.4
+cbor2==6.1.3
+certifi==2026.6.17
+cffi==2.1.0
+charset-normalizer==3.4.9
+click==8.4.2
+cloudpickle==3.1.2
+compressed-tensors==0.15.0.1
+cryptography==49.0.0
+cuda-bindings==12.9.7
+cuda-pathfinder==1.5.6
+cuda-python==12.9.7
+cuda-tile==1.5.0
+cuda-toolkit==12.9.1
+depyf==0.20.0
+detect-installer==0.1.0
+dill==0.4.1
+diskcache==5.6.3
+distro==1.9.0
+dnspython==2.8.0
+docstring-parser==0.18.0
+einops==0.8.2
+email-validator==2.3.0
+fastapi==0.139.2
+fastapi-cli==0.0.32
+fastapi-cloud-cli==0.22.2
+fastar==0.11.0
+fastsafetensors==0.3.3
+filelock==3.31.1
+flashinfer-cubin==0.6.8.post1
+flashinfer-python==0.6.8.post1
+frozenlist==1.8.0
+fsspec==2026.6.0
+gguf==0.19.0
+googleapis-common-protos==1.75.0
+grpcio==1.82.1
+h11==0.16.0
+hf-xet==1.5.2
+httpcore==1.0.9
+httptools==0.8.0
+httpx==0.28.1
+httpx-sse==0.4.3
+huggingface-hub==1.24.0
+idna==3.18
+ijson==3.5.1
+interegular==0.3.3
+jinja2==3.1.6
+jiter==0.16.0
+jmespath==1.1.0
+jsonschema==4.26.0
+jsonschema-specifications==2025.9.1
+lark==1.2.2
+llguidance==1.3.0
+llvmlite==0.47.0
+lm-format-enforcer==0.11.3
+loguru==0.7.3
+markdown-it-py==4.2.0
+markupsafe==3.0.3
+mcp==1.28.1
+mdurl==0.1.2
+mistral-common==1.11.6
+ml-dtypes==0.5.4
+model-hosting-container-standards==0.1.16
+mpmath==1.3.0
+msgspec==0.21.1
+multidict==6.7.1
+networkx==3.6.1
+ninja==1.13.0
+numba==0.65.0
+numpy==2.3.5
+nvidia-cublas-cu12==12.9.1.4
+nvidia-cuda-cupti-cu12==12.9.79
+nvidia-cuda-nvrtc-cu12==12.9.86
+nvidia-cuda-runtime-cu12==12.9.79
+nvidia-cudnn-cu12==9.17.1.4
+nvidia-cudnn-frontend==1.18.0
+nvidia-cufft-cu12==11.4.1.4
+nvidia-cufile-cu12==1.14.1.1
+nvidia-curand-cu12==10.3.10.19
+nvidia-cusolver-cu12==11.7.5.82
+nvidia-cusparse-cu12==12.5.10.65
+nvidia-cusparselt-cu12==0.7.1
+nvidia-cutlass-dsl==4.5.3
+nvidia-cutlass-dsl-libs-base==4.5.3
+nvidia-ml-py==13.610.43
+nvidia-nccl-cu12==2.28.9
+nvidia-nvjitlink-cu12==12.9.86
+nvidia-nvshmem-cu12==3.4.5
+nvidia-nvtx-cu12==12.9.79
+openai==2.46.0
+openai-harmony==0.0.8
+opencv-python-headless==5.0.0.93
+opentelemetry-api==1.44.0
+opentelemetry-exporter-otlp==1.44.0
+opentelemetry-exporter-otlp-proto-common==1.44.0
+opentelemetry-exporter-otlp-proto-grpc==1.44.0
+opentelemetry-exporter-otlp-proto-http==1.44.0
+opentelemetry-proto==1.44.0
+opentelemetry-sdk==1.44.0
+opentelemetry-semantic-conventions==0.65b0
+opentelemetry-semantic-conventions-ai==0.5.1
+outlines-core==0.2.14
+packaging==26.2
+partial-json-parser==0.2.1.1.post7
+pillow==12.3.0
+prometheus-client==0.25.0
+prometheus-fastapi-instrumentator==8.0.2
+propcache==0.5.2
+protobuf==7.35.1
+psutil==7.2.2
+py-cpuinfo==9.0.0
+pybase64==1.4.3
+pycountry==26.2.16
+pycparser==3.0
+pydantic==2.13.4
+pydantic-core==2.46.4
+pydantic-extra-types==2.11.1
+pydantic-settings==2.14.2
+pygments==2.20.0
+pyjwt==2.13.0
+python-dotenv==1.2.2
+python-json-logger==4.1.0
+python-multipart==0.0.32
+pyyaml==6.0.3
+pyzmq==27.1.0
+quack-kernels==0.5.0
+referencing==0.37.0
+regex==2026.7.19
+requests==2.34.2
+rich==15.0.0
+rich-toolkit==0.20.3
+rignore==0.8.0
+rpds-py==2026.6.3
+safetensors==0.8.0
+sentencepiece==0.2.2
+sentry-sdk==2.66.0
+setproctitle==1.3.7
+setuptools==80.10.2
+shellingham==1.5.4
+six==1.17.0
+sniffio==1.3.1
+sse-starlette==3.4.5
+starlette==1.3.1
+supervisor==4.3.0
+sympy==1.14.0
+tabulate==0.10.0
+tiktoken==0.13.0
+tilelang==0.1.9
+tokenizers==0.22.2
+torch==2.11.0+cu129
+torch-c-dlpack-ext==0.1.5
+torchaudio==2.11.0+cu129
+torchvision==0.26.0+cu129
+tqdm==4.69.0
+transformers==5.14.1
+triton==3.6.0
+typer==0.27.0
+typing-extensions==4.16.0
+typing-inspection==0.4.2
+urllib3==2.7.0
+uvicorn==0.51.0
+uvloop==0.22.1
+vllm @ https://github.com/vllm-project/vllm/releases/download/v0.20.0/vllm-0.20.0%2Bcu129-cp38-abi3-manylinux_2_31_x86_64.whl
+watchfiles==1.2.0
+websockets==16.1.1
+xgrammar==0.2.3
+yarl==1.24.5
+z3-solver==4.15.4.0

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/source.sha256

@@ -0,0 +1,2 @@
+42381f4bd1b67710d4068eb8993240930326a804c6e5167b0b78df3420e47c74  /home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/../frontier-qwen30-vllm020-profile-v1/profile_vllm020_flashattn.py
+8a7a72b3adb25db6a61a86f9b04d2d19ebbfe3f2d20480852ef2f45376ac3b85  runs/frontier-code-trace-v0/run_flashattn_code_longctx.sh

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/vllm-allow-long-max-model-len.txt

@@ -0,0 +1 @@
+1

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/provenance/vllm-source.commit

@@ -0,0 +1 @@
+88d34c6409e9fb3c7b8ca0c04756f061d2099eb1

runs/frontier-code-trace-v0/profiles/raw/tp4-r2-v2/raw/flashattn-code-longctx-tp4.json

@@ -0,0 +1,564 @@
+{
+  "environment": {
+    "attention_backend": "FLASH_ATTN",
+    "block_size": 16,
+    "dtype": "bfloat16",
+    "gpu": "NVIDIA H20",
+    "model": "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B",
+    "profile_kv_update": true,
+    "profile_method": "cuda_event",
+    "torch_cuda": "12.9",
+    "torch_version": "2.11.0+cu129",
+    "vllm_source_commit": "88d34c6409e9fb3c7b8ca0c04756f061d2099eb1",
+    "vllm_version": "0.20.0"
+  },
+  "rows": [
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks48k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.040991999208927155,
+        "mean_ms": 0.02782719973474741,
+        "median_ms": 0.025295999832451344,
+        "min_ms": 0.02412799932062626,
+        "std_ms": 0.005434033125981124
+      },
+      "max_time": 0.011621503829956055,
+      "mean_time": 0.011469862461090087,
+      "memory_allocated_mb": 60.0849609375,
+      "memory_reserved_mb": 62.0,
+      "min_time": 0.01142630386352539,
+      "std_time": 5.2644635131708985e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 714219.5495185945
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks64k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03811199963092804,
+        "mean_ms": 0.02837119996547699,
+        "median_ms": 0.026176000013947487,
+        "min_ms": 0.024159999564290047,
+        "std_ms": 0.004902838828525061
+      },
+      "max_time": 0.015688991546630858,
+      "mean_time": 0.015603910255432129,
+      "memory_allocated_mb": 68.0888671875,
+      "memory_reserved_mb": 94.0,
+      "min_time": 0.01557091236114502,
+      "std_time": 3.431861179521254e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 524996.6108429873
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks80k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.029759999364614487,
+        "mean_ms": 0.02612799983471632,
+        "median_ms": 0.025695999152958393,
+        "min_ms": 0.024000000208616257,
+        "std_ms": 0.0019206532069484194
+      },
+      "max_time": 0.020114784240722657,
+      "mean_time": 0.019786701011657717,
+      "memory_allocated_mb": 76.0927734375,
+      "memory_reserved_mb": 134.0,
+      "min_time": 0.019717824935913085,
+      "std_time": 0.00011854565766692264,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 414015.4538734641
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks96k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.032896000891923904,
+        "mean_ms": 0.026428800262510776,
+        "median_ms": 0.02483200002461672,
+        "min_ms": 0.024159999564290047,
+        "std_ms": 0.0028081875041272375
+      },
+      "max_time": 0.023987648010253906,
+      "mean_time": 0.023885836791992184,
+      "memory_allocated_mb": 84.0966796875,
+      "memory_reserved_mb": 182.0,
+      "min_time": 0.023849760055541992,
+      "std_time": 4.082290087440645e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 342964.7481618228
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks112k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03388800099492073,
+        "mean_ms": 0.02606080025434494,
+        "median_ms": 0.02440000046044588,
+        "min_ms": 0.023840000852942467,
+        "std_ms": 0.003071273491702287
+      },
+      "max_time": 0.02814064025878906,
+      "mean_time": 0.028014582252502435,
+      "memory_allocated_mb": 92.1005859375,
+      "memory_reserved_mb": 238.0,
+      "min_time": 0.027969343185424805,
+      "std_time": 4.739888751775852e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 292419.138224638
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.02953599952161312,
+        "mean_ms": 0.025788800045847892,
+        "median_ms": 0.024800000712275505,
+        "min_ms": 0.024191999807953835,
+        "std_ms": 0.002060583463186555
+      },
+      "max_time": 0.03218783950805664,
+      "mean_time": 0.03215217628479004,
+      "memory_allocated_mb": 100.1044921875,
+      "memory_reserved_mb": 302.0,
+      "min_time": 0.03213116836547852,
+      "std_time": 1.9548439899050275e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 254788.35172583075
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q8ks136k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.032735999673604965,
+        "mean_ms": 0.02950399983674288,
+        "median_ms": 0.028351999819278717,
+        "min_ms": 0.027904000133275986,
+        "std_ms": 0.0017488521169760844
+      },
+      "max_time": 0.03428476715087891,
+      "mean_time": 0.03422174110412597,
+      "memory_allocated_mb": 104.1064453125,
+      "memory_reserved_mb": 370.0,
+      "min_time": 0.034186080932617186,
+      "std_time": 3.324003475365458e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 239379.98873506542
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s128k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03446400165557861,
+        "mean_ms": 0.0248607998713851,
+        "median_ms": 0.023519999347627163,
+        "min_ms": 0.0226879995316267,
+        "std_ms": 0.0035075699446123015
+      },
+      "max_time": 0.003272864103317261,
+      "mean_time": 0.003240902423858643,
+      "memory_allocated_mb": 66.2958984375,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.003228480100631714,
+      "std_time": 1.4700871477091224e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 157980.68964705482
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q2ks66k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.03484800085425377,
+        "mean_ms": 0.02619840018451214,
+        "median_ms": 0.024639999493956566,
+        "min_ms": 0.02284800074994564,
+        "std_ms": 0.0037605668537262792
+      },
+      "max_time": 0.005165311813354492,
+      "mean_time": 0.004916048002243043,
+      "memory_allocated_mb": 42.04248046875,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.004875135898590088,
+      "std_time": 8.365277131314812e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 416594.79302593466
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q4ks100k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.023615999147295952,
+        "mean_ms": 0.018966399878263474,
+        "median_ms": 0.01788800023496151,
+        "min_ms": 0.01679999940097332,
+        "std_ms": 0.0021688374822942504
+      },
+      "max_time": 0.0145830717086792,
+      "mean_time": 0.01451537265777588,
+      "memory_allocated_mb": 68.06640625,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.014502431869506836,
+      "std_time": 2.2952325515836057e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 282183.5922900522
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q6ks134k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.028192000463604927,
+        "mean_ms": 0.021894400380551814,
+        "median_ms": 0.021088000386953354,
+        "min_ms": 0.020640000700950623,
+        "std_ms": 0.0021379161656161685
+      },
+      "max_time": 0.024349727630615235,
+      "mean_time": 0.024299632072448733,
+      "memory_allocated_mb": 96.09033203125,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.024284608840942384,
+      "std_time": 2.0842542550131525e-05,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 252843.3344867865
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q1ks8k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.0261439997702837,
+        "mean_ms": 0.016019200067967178,
+        "median_ms": 0.01512000011280179,
+        "min_ms": 0.014368000440299511,
+        "std_ms": 0.0034013214299104646
+      },
+      "max_time": 0.0003514559864997864,
+      "mean_time": 0.00034397439956665037,
+      "memory_allocated_mb": 8.5205078125,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.0003375680148601532,
+      "std_time": 3.760143706773439e-06,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 2976965.731432534
+    },
+    {
+      "attention_core_excludes_kv_cache_update": true,
+      "config": {
+        "backend": "FLASH_ATTN",
+        "batch_spec": "q512s4k",
+        "block_size": 16,
+        "device": "cuda:0",
+        "dtype": "torch.bfloat16",
+        "head_dim": 128,
+        "kv_cache_dtype": "auto",
+        "kv_lora_rank": null,
+        "num_kv_heads": 1,
+        "num_kv_splits": null,
+        "num_layers": 1,
+        "num_q_heads": 8,
+        "prefill_backend": null,
+        "profile_memory": true,
+        "qk_nope_head_dim": null,
+        "qk_rope_head_dim": null,
+        "reorder_batch_threshold": null,
+        "repeats": 10,
+        "use_cuda_graphs": false,
+        "v_head_dim": null,
+        "warmup_iters": 5
+      },
+      "error": null,
+      "kv_cache_update_time": {
+        "max_ms": 0.021023999899625778,
+        "mean_ms": 0.015327999833971262,
+        "median_ms": 0.014639999717473984,
+        "min_ms": 0.013856000266969204,
+        "std_ms": 0.0020222886267191407
+      },
+      "max_time": 0.0001605760008096695,
+      "mean_time": 0.0001493696004152298,
+      "memory_allocated_mb": 4.265625,
+      "memory_reserved_mb": 372.0,
+      "min_time": 0.00014313599467277526,
+      "std_time": 5.429351067501872e-06,
+      "tensor_parallel_size": 4,
+      "throughput_tokens_per_sec": 3427738.9681481416
+    }
+  ],
+  "schema_version": "qwen30_vllm020_flashattn_raw.v1"
+}

runs/frontier-code-trace-v0/profiles/repeat-check.json

@@ -0,0 +1,320 @@
+{
+  "comparisons": [
+    {
+      "batch_spec": "q1ks8k",
+      "first_mean_s": 0.0011425888061523438,
+      "pass": true,
+      "relative_difference": 0.0011153009347107706,
+      "second_mean_s": 0.001141315186023712,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q2ks66k",
+      "first_mean_s": 0.01678766403198242,
+      "pass": true,
+      "relative_difference": 0.0007022802901221452,
+      "second_mean_s": 0.016775878524780276,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q4ks100k",
+      "first_mean_s": 0.05035054740905761,
+      "pass": true,
+      "relative_difference": 0.000388526368934946,
+      "second_mean_s": 0.0503309886932373,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q512s128k",
+      "first_mean_s": 0.009489968109130859,
+      "pass": true,
+      "relative_difference": 0.005183356056904404,
+      "second_mean_s": 0.009440905380249024,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q512s4k",
+      "first_mean_s": 0.0003366623997688293,
+      "pass": true,
+      "relative_difference": 0.0014267930704769689,
+      "second_mean_s": 0.0003361823946237564,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q6ks134k",
+      "first_mean_s": 0.0961898666381836,
+      "pass": true,
+      "relative_difference": 0.0005591912667149224,
+      "second_mean_s": 0.09613609313964844,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q8ks112k",
+      "first_mean_s": 0.1069560287475586,
+      "pass": true,
+      "relative_difference": 0.00031876960426212737,
+      "second_mean_s": 0.10692193984985351,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q8ks128k",
+      "first_mean_s": 0.12283342437744141,
+      "pass": true,
+      "relative_difference": 0.0004249343895711351,
+      "second_mean_s": 0.12278123931884766,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q8ks136k",
+      "first_mean_s": 0.13079229431152345,
+      "pass": true,
+      "relative_difference": 0.0005674643646600315,
+      "second_mean_s": 0.1307180953979492,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q8ks48k",
+      "first_mean_s": 0.04368390693664551,
+      "pass": true,
+      "relative_difference": 0.0004927566415802605,
+      "second_mean_s": 0.043662386703491214,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q8ks64k",
+      "first_mean_s": 0.0595021312713623,
+      "pass": true,
+      "relative_difference": 0.0004229478941120101,
+      "second_mean_s": 0.0594769702911377,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q8ks80k",
+      "first_mean_s": 0.07538758392333984,
+      "pass": true,
+      "relative_difference": 0.0012547739275033378,
+      "second_mean_s": 0.07529304885864258,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q8ks96k",
+      "first_mean_s": 0.09124225158691406,
+      "pass": true,
+      "relative_difference": 0.0015351127502189047,
+      "second_mean_s": 0.09110229187011719,
+      "tp": 1
+    },
+    {
+      "batch_spec": "q1ks8k",
+      "first_mean_s": 0.0006064576089382171,
+      "pass": true,
+      "relative_difference": 0.04648308449369336,
+      "second_mean_s": 0.0006353183984756471,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q2ks66k",
+      "first_mean_s": 0.00959802885055542,
+      "pass": true,
+      "relative_difference": 0.003767189056248553,
+      "second_mean_s": 0.009561939239501951,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q4ks100k",
+      "first_mean_s": 0.025250255966186526,
+      "pass": true,
+      "relative_difference": 0.001402319427652538,
+      "second_mean_s": 0.0252856897354126,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q512s128k",
+      "first_mean_s": 0.004790303993225097,
+      "pass": true,
+      "relative_difference": 0.006859924072500961,
+      "second_mean_s": 0.0047575551986694335,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q512s4k",
+      "first_mean_s": 0.00020788159817457198,
+      "pass": true,
+      "relative_difference": 0.008626540222728182,
+      "second_mean_s": 0.00020609600096940998,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q6ks134k",
+      "first_mean_s": 0.048172192001342776,
+      "pass": true,
+      "relative_difference": 0.0005976896073522018,
+      "second_mean_s": 0.048143408584594725,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q8ks112k",
+      "first_mean_s": 0.05542207336425782,
+      "pass": true,
+      "relative_difference": 0.0006067719206058191,
+      "second_mean_s": 0.055455712127685554,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q8ks128k",
+      "first_mean_s": 0.06362084197998047,
+      "pass": true,
+      "relative_difference": 6.421681207537411e-06,
+      "second_mean_s": 0.06362125053405762,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q8ks136k",
+      "first_mean_s": 0.06770619888305665,
+      "pass": true,
+      "relative_difference": 0.00015497334523802088,
+      "second_mean_s": 0.06771669235229492,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q8ks48k",
+      "first_mean_s": 0.02263068161010742,
+      "pass": true,
+      "relative_difference": 0.003557316307676039,
+      "second_mean_s": 0.022550320053100585,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q8ks64k",
+      "first_mean_s": 0.03074285774230957,
+      "pass": true,
+      "relative_difference": 0.0007706865344215391,
+      "second_mean_s": 0.030766559982299803,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q8ks80k",
+      "first_mean_s": 0.038945382690429686,
+      "pass": true,
+      "relative_difference": 0.001695543775275234,
+      "second_mean_s": 0.03901147232055664,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q8ks96k",
+      "first_mean_s": 0.04714819869995117,
+      "pass": true,
+      "relative_difference": 0.0014946162280953233,
+      "second_mean_s": 0.0472187198638916,
+      "tp": 2
+    },
+    {
+      "batch_spec": "q1ks8k",
+      "first_mean_s": 0.0003469599992036819,
+      "pass": true,
+      "relative_difference": 0.008642208702722183,
+      "second_mean_s": 0.00034397439956665037,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q2ks66k",
+      "first_mean_s": 0.004817302370071412,
+      "pass": true,
+      "relative_difference": 0.020290162871872575,
+      "second_mean_s": 0.004916048002243043,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q4ks100k",
+      "first_mean_s": 0.014412275123596191,
+      "pass": true,
+      "relative_difference": 0.007127958343441791,
+      "second_mean_s": 0.01451537265777588,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q512s128k",
+      "first_mean_s": 0.003182867193222046,
+      "pass": true,
+      "relative_difference": 0.018068901625077752,
+      "second_mean_s": 0.003240902423858643,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q512s4k",
+      "first_mean_s": 0.00015103680044412614,
+      "pass": true,
+      "relative_difference": 0.011099630528025208,
+      "second_mean_s": 0.0001493696004152298,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q6ks134k",
+      "first_mean_s": 0.02411243553161621,
+      "pass": true,
+      "relative_difference": 0.007733466059061839,
+      "second_mean_s": 0.024299632072448733,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q8ks112k",
+      "first_mean_s": 0.027747158622741696,
+      "pass": true,
+      "relative_difference": 0.009591652827304903,
+      "second_mean_s": 0.028014582252502435,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q8ks128k",
+      "first_mean_s": 0.03186994876861572,
+      "pass": true,
+      "relative_difference": 0.008816561960069082,
+      "second_mean_s": 0.03215217628479004,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q8ks136k",
+      "first_mean_s": 0.033897081375122075,
+      "pass": true,
+      "relative_difference": 0.009532159165047292,
+      "second_mean_s": 0.03422174110412597,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q8ks48k",
+      "first_mean_s": 0.011354345512390137,
+      "pass": true,
+      "relative_difference": 0.010122318271387163,
+      "second_mean_s": 0.011469862461090087,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q8ks64k",
+      "first_mean_s": 0.015451993656158448,
+      "pass": true,
+      "relative_difference": 0.00978342795663941,
+      "second_mean_s": 0.015603910255432129,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q8ks80k",
+      "first_mean_s": 0.01954985942840576,
+      "pass": true,
+      "relative_difference": 0.012041804397861736,
+      "second_mean_s": 0.019786701011657717,
+      "tp": 4
+    },
+    {
+      "batch_spec": "q8ks96k",
+      "first_mean_s": 0.02367282257080078,
+      "pass": true,
+      "relative_difference": 0.008957957353947301,
+      "second_mean_s": 0.023885836791992184,
+      "tp": 4
+    }
+  ],
+  "maximum_relative_difference": 0.04648308449369336,
+  "schema": "frontier-code-longctx-repeat-check-v1",
+  "status": "PASS",
+  "threshold": 0.05
+}

runs/frontier-code-trace-v0/results/calibration-summary.json

@@ -0,0 +1,634 @@
+{
+  "attention_profile_sha256": "fbcf7e1f95789a6f6d771e24d1fc60958b7daf04eb0db260d27869a19d71d550",
+  "cell_version": "v3",
+  "cells": [
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 5525
+        },
+        "max": 1,
+        "share_gt_1": 0.0,
+        "share_gt_4": 0.0,
+        "stages": 5525
+      },
+      "drain_fraction": 0.00044126232317770095,
+      "drain_tail_s": 1.6150201028303854,
+      "offered_load": {
+        "decode_tokens_per_s": 1.5183060109289617,
+        "prefill_tokens_per_s_after_prefix": 189.91639344262296,
+        "prefill_tokens_per_s_raw": 201.38743169398907,
+        "requests_per_s": 0.008743169398907104
+      },
+      "prefix_cache_hit_ratio": 0.05698154180238871,
+      "requests": 32,
+      "rho": 5e-05,
+      "subcritical_gate": true,
+      "topology": "tp4_mns16",
+      "tpot_ms": {
+        "count": 32,
+        "max": 6.06516500761245,
+        "mean": 5.267661756941212,
+        "min": 4.642776716764274,
+        "p50": 5.27961358181982,
+        "p90": 6.065165007589712,
+        "p95": 6.06516500761245,
+        "p99": 6.06516500761245
+      },
+      "trace_sha256": "bdc54edafacf3e3f353de7a54d5e43a7e4323eb1e8b3dc55ce333a5e0b347ce5",
+      "ttft_ms": {
+        "count": 32,
+        "max": 9026.404118319988,
+        "mean": 1316.2539772416199,
+        "min": 76.98844366404956,
+        "p50": 401.828087941567,
+        "p90": 3363.622890573971,
+        "p95": 6721.082190563591,
+        "p99": 8741.382807755395
+      },
+      "waiting_ms": {
+        "count": 32,
+        "max": 0.0,
+        "mean": 0.0,
+        "min": 0.0,
+        "p50": 0.0,
+        "p90": 0.0,
+        "p95": 0.0,
+        "p99": 0.0
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 33249,
+          "2": 294,
+          "3": 76
+        },
+        "max": 3,
+        "share_gt_1": 0.011005681311163331,
+        "share_gt_4": 0.0,
+        "stages": 33619
+      },
+      "drain_fraction": 0.0006573023445132039,
+      "drain_tail_s": 2.4057265809183264,
+      "offered_load": {
+        "decode_tokens_per_s": 9.343989071038251,
+        "prefill_tokens_per_s_after_prefix": 491.8571038251366,
+        "prefill_tokens_per_s_raw": 1477.1773224043716,
+        "requests_per_s": 0.03224043715846994
+      },
+      "prefix_cache_hit_ratio": 0.667142227102801,
+      "requests": 118,
+      "rho": 0.0001,
+      "subcritical_gate": true,
+      "topology": "tp4_mns16",
+      "tpot_ms": {
+        "count": 118,
+        "max": 109.2514556116484,
+        "mean": 6.521761912064516,
+        "min": 4.642776716750063,
+        "p50": 5.7134254039277,
+        "p90": 6.06516500761245,
+        "p95": 6.582261050425584,
+        "p99": 14.049063410907634
+      },
+      "trace_sha256": "8d4ac5646f46e84560c17c44013af2bcb7cfe91d4a2be3476d8f3c09565cf55f",
+      "ttft_ms": {
+        "count": 118,
+        "max": 9696.373176562247,
+        "mean": 1203.6180593647337,
+        "min": 72.1560867923472,
+        "p50": 592.0461311603731,
+        "p90": 3231.0798294979118,
+        "p95": 6885.471905257904,
+        "p99": 8918.025104477005
+      },
+      "waiting_ms": {
+        "count": 118,
+        "max": 7151.845512636555,
+        "mean": 98.59009397263883,
+        "min": 0.0,
+        "p50": 0.0,
+        "p90": 0.0,
+        "p95": 0.21863999056675887,
+        "p99": 2087.1951396751865
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 71338,
+          "2": 4093,
+          "3": 1021,
+          "4": 33
+        },
+        "max": 4,
+        "share_gt_1": 0.067294240700791,
+        "share_gt_4": 0.0,
+        "stages": 76485
+      },
+      "drain_fraction": 0.001483881651363999,
+      "drain_tail_s": 5.431006843992236,
+      "offered_load": {
+        "decode_tokens_per_s": 22.691803278688525,
+        "prefill_tokens_per_s_after_prefix": 884.2945355191257,
+        "prefill_tokens_per_s_raw": 2459.7437158469947,
+        "requests_per_s": 0.0633879781420765
+      },
+      "prefix_cache_hit_ratio": 0.6406200282639031,
+      "requests": 232,
+      "rho": 0.0002,
+      "subcritical_gate": true,
+      "topology": "tp2_mns16",
+      "tpot_ms": {
+        "count": 231,
+        "max": 151.0151050020596,
+        "mean": 8.197384891414327,
+        "min": 4.790955988028145,
+        "p50": 6.377284733844135,
+        "p90": 9.066414456096654,
+        "p95": 13.235755136549969,
+        "p99": 45.72584194314243
+      },
+      "trace_sha256": "188f9e52674ec268d70d763995f2cdf84ccbcbab86420214b127994569324e9b",
+      "ttft_ms": {
+        "count": 232,
+        "max": 22394.384315265255,
+        "mean": 2036.2309970727358,
+        "min": 90.95485497891787,
+        "p50": 876.6052006358223,
+        "p90": 5676.270318256946,
+        "p95": 9955.05948414866,
+        "p99": 15641.83197487735
+      },
+      "waiting_ms": {
+        "count": 232,
+        "max": 13155.607757982125,
+        "mean": 338.8558697712575,
+        "min": 0.0,
+        "p50": 0.0,
+        "p90": 5.941389865392926,
+        "p95": 1644.4218195787325,
+        "p99": 8012.825782587187
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 72379,
+          "2": 4138,
+          "3": 696
+        },
+        "max": 3,
+        "share_gt_1": 0.06260603784336835,
+        "share_gt_4": 0.0,
+        "stages": 77213
+      },
+      "drain_fraction": 0.0010192168060329308,
+      "drain_tail_s": 3.730333510080527,
+      "offered_load": {
+        "decode_tokens_per_s": 22.691803278688525,
+        "prefill_tokens_per_s_after_prefix": 880.7972677595628,
+        "prefill_tokens_per_s_raw": 2459.7437158469947,
+        "requests_per_s": 0.0633879781420765
+      },
+      "prefix_cache_hit_ratio": 0.6420421114379927,
+      "requests": 232,
+      "rho": 0.0002,
+      "subcritical_gate": true,
+      "topology": "tp4_mns16",
+      "tpot_ms": {
+        "count": 231,
+        "max": 109.2514556116484,
+        "mean": 6.682483450713786,
+        "min": 4.6427767165369005,
+        "p50": 5.757631515507455,
+        "p90": 6.873691941109428,
+        "p95": 8.022302513342353,
+        "p99": 28.619774468019706
+      },
+      "trace_sha256": "188f9e52674ec268d70d763995f2cdf84ccbcbab86420214b127994569324e9b",
+      "ttft_ms": {
+        "count": 232,
+        "max": 12080.695954866542,
+        "mean": 1109.6819359234096,
+        "min": 72.1560867923472,
+        "p50": 527.4056752296019,
+        "p90": 2603.3383651315494,
+        "p95": 5585.788369509169,
+        "p99": 8828.771798959253
+      },
+      "waiting_ms": {
+        "count": 232,
+        "max": 7151.845512636555,
+        "mean": 100.5913696184332,
+        "min": 0.0,
+        "p50": 0.0,
+        "p90": 2.71668797665825,
+        "p95": 337.7136230834021,
+        "p99": 2261.809117635907
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 163554,
+          "10": 18,
+          "11": 5,
+          "12": 10,
+          "2": 48104,
+          "3": 15894,
+          "4": 2673,
+          "5": 1305,
+          "6": 332,
+          "7": 181,
+          "8": 46,
+          "9": 15
+        },
+        "max": 12,
+        "share_gt_1": 0.2954419157652593,
+        "share_gt_4": 0.008236515505929689,
+        "stages": 232137
+      },
+      "drain_fraction": 0.001932022033012745,
+      "drain_tail_s": 7.071200640826646,
+      "offered_load": {
+        "decode_tokens_per_s": 90.30054644808743,
+        "prefill_tokens_per_s_after_prefix": 1805.1767759562842,
+        "prefill_tokens_per_s_raw": 5620.918852459016,
+        "requests_per_s": 0.13633879781420766
+      },
+      "prefix_cache_hit_ratio": 0.6789729811301968,
+      "requests": 499,
+      "rho": 0.0004,
+      "subcritical_gate": true,
+      "topology": "tp2_mns16",
+      "tpot_ms": {
+        "count": 497,
+        "max": 556.5990343567938,
+        "mean": 18.61788167095804,
+        "min": 4.790955988028145,
+        "p50": 7.918935117459114,
+        "p90": 26.775158132602897,
+        "p95": 56.43467403678585,
+        "p99": 256.82570583636533
+      },
+      "trace_sha256": "2751c373bb91eccf42b94e1b28245870d6ffa2f91e1b5dabafe2435bc46da274",
+      "ttft_ms": {
+        "count": 499,
+        "max": 28488.652032648133,
+        "mean": 2601.050597929721,
+        "min": 90.96449664048123,
+        "p50": 942.4448686800133,
+        "p90": 7645.075071658986,
+        "p95": 12524.265102993284,
+        "p99": 19300.755134457228
+      },
+      "waiting_ms": {
+        "count": 499,
+        "max": 25277.458416518584,
+        "mean": 926.4084886277059,
+        "min": 0.0,
+        "p50": 2.7291107814733095,
+        "p90": 2483.439910098784,
+        "p95": 6352.661668917902,
+        "p99": 13692.62653031437
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 184431,
+          "2": 52296,
+          "3": 10583,
+          "4": 1254,
+          "5": 675
+        },
+        "max": 5,
+        "share_gt_1": 0.2600235115692167,
+        "share_gt_4": 0.002708243894414598,
+        "stages": 249239
+      },
+      "drain_fraction": 0.001114813580178276,
+      "drain_tail_s": 4.08021770345249,
+      "offered_load": {
+        "decode_tokens_per_s": 90.30054644808743,
+        "prefill_tokens_per_s_after_prefix": 1738.4489071038251,
+        "prefill_tokens_per_s_raw": 5620.918852459016,
+        "requests_per_s": 0.13633879781420766
+      },
+      "prefix_cache_hit_ratio": 0.6908465352465023,
+      "requests": 499,
+      "rho": 0.0004,
+      "subcritical_gate": true,
+      "topology": "tp4_mns16",
+      "tpot_ms": {
+        "count": 497,
+        "max": 191.36693232402044,
+        "mean": 9.29907499058627,
+        "min": 4.6427767165369005,
+        "p50": 6.554626532726112,
+        "p90": 10.321323503714908,
+        "p95": 18.835821286381098,
+        "p99": 73.17188748931419
+      },
+      "trace_sha256": "2751c373bb91eccf42b94e1b28245870d6ffa2f91e1b5dabafe2435bc46da274",
+      "ttft_ms": {
+        "count": 499,
+        "max": 14944.866787426236,
+        "mean": 1252.450784907019,
+        "min": 72.1560867923472,
+        "p50": 526.7327146962089,
+        "p90": 3295.008250609953,
+        "p95": 5487.844406213842,
+        "p99": 10840.596450182773
+      },
+      "waiting_ms": {
+        "count": 499,
+        "max": 12922.960017376226,
+        "mean": 274.43276731097365,
+        "min": 0.0,
+        "p50": 0.6520015415389935,
+        "p90": 164.9360333363801,
+        "p95": 2029.720098907419,
+        "p99": 6246.634668026156
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 123423,
+          "10": 262,
+          "11": 84,
+          "12": 37,
+          "13": 42,
+          "14": 108,
+          "15": 59,
+          "16": 39,
+          "2": 64628,
+          "3": 34551,
+          "4": 17977,
+          "5": 5215,
+          "6": 2255,
+          "7": 1043,
+          "8": 349,
+          "9": 406
+        },
+        "max": 16,
+        "share_gt_1": 0.5072501377366475,
+        "share_gt_4": 0.039520436924600166,
+        "stages": 250478
+      },
+      "drain_fraction": 0.0006878766017147268,
+      "drain_tail_s": 2.5176283622759,
+      "offered_load": {
+        "decode_tokens_per_s": 135.20737704918034,
+        "prefill_tokens_per_s_after_prefix": 3107.11174863388,
+        "prefill_tokens_per_s_raw": 7722.316120218579,
+        "requests_per_s": 0.23743169398907105
+      },
+      "prefix_cache_hit_ratio": 0.5977800565206239,
+      "requests": 869,
+      "rho": 0.0008,
+      "subcritical_gate": true,
+      "topology": "tp2_mns16",
+      "tpot_ms": {
+        "count": 864,
+        "max": 1048.199917263105,
+        "mean": 35.531282784807445,
+        "min": 4.790955988028145,
+        "p50": 10.56810791578755,
+        "p90": 58.679343047855866,
+        "p95": 113.21830549665293,
+        "p99": 497.8300148864786
+      },
+      "trace_sha256": "2788a21c6bb8be6d03fb5151f2c51efa09c0eb5e1f941af60c00a7cc11d41206",
+      "ttft_ms": {
+        "count": 869,
+        "max": 35023.43713844812,
+        "mean": 3519.677566968775,
+        "min": 80.48812343014333,
+        "p50": 944.5946905507299,
+        "p90": 11539.678186253443,
+        "p95": 16395.861888973763,
+        "p99": 23283.982779742393
+      },
+      "waiting_ms": {
+        "count": 869,
+        "max": 33885.372635082604,
+        "mean": 1944.5601581730987,
+        "min": 0.0,
+        "p50": 4.989910657513974,
+        "p90": 7660.373377703809,
+        "p95": 12268.558603129362,
+        "p99": 19104.247337156932
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 185761,
+          "2": 81136,
+          "3": 28184,
+          "4": 10328,
+          "5": 2183,
+          "6": 759,
+          "7": 199,
+          "8": 89,
+          "9": 18
+        },
+        "max": 9,
+        "share_gt_1": 0.3981636573931581,
+        "share_gt_4": 0.010523007739983218,
+        "stages": 308657
+      },
+      "drain_fraction": 0.0005799250803688739,
+      "drain_tail_s": 2.1225257941500786,
+      "offered_load": {
+        "decode_tokens_per_s": 135.20737704918034,
+        "prefill_tokens_per_s_after_prefix": 2943.3609289617484,
+        "prefill_tokens_per_s_raw": 7722.316120218579,
+        "requests_per_s": 0.23743169398907105
+      },
+      "prefix_cache_hit_ratio": 0.6189897292366545,
+      "requests": 869,
+      "rho": 0.0008,
+      "subcritical_gate": true,
+      "topology": "tp4_mns16",
+      "tpot_ms": {
+        "count": 864,
+        "max": 556.1359991102521,
+        "mean": 12.147948538437653,
+        "min": 4.6427767165369005,
+        "p50": 6.843951976861717,
+        "p90": 14.306706048821074,
+        "p95": 30.43280397112956,
+        "p99": 137.6170979713351
+      },
+      "trace_sha256": "2788a21c6bb8be6d03fb5151f2c51efa09c0eb5e1f941af60c00a7cc11d41206",
+      "ttft_ms": {
+        "count": 869,
+        "max": 17076.806859823137,
+        "mean": 1363.8187082248587,
+        "min": 67.46058986817616,
+        "p50": 457.5338581378219,
+        "p90": 4063.5681271006715,
+        "p95": 6757.6291299215145,
+        "p99": 11100.99635793714
+      },
+      "waiting_ms": {
+        "count": 869,
+        "max": 16033.766590147934,
+        "mean": 459.6734459851503,
+        "min": 0.0,
+        "p50": 2.5305451699750847,
+        "p90": 796.2202857544993,
+        "p95": 3149.129404067843,
+        "p99": 9085.03332971697
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 8883,
+          "10": 102,
+          "11": 206,
+          "12": 48,
+          "13": 104,
+          "14": 98,
+          "15": 112,
+          "16": 42432,
+          "2": 8887,
+          "3": 5320,
+          "4": 1084,
+          "5": 391,
+          "6": 1238,
+          "7": 983,
+          "8": 600,
+          "9": 185
+        },
+        "max": 16,
+        "share_gt_1": 0.8743084346214255,
+        "share_gt_4": 0.6579457501450342,
+        "stages": 70673
+      },
+      "drain_fraction": 0.26690162220748465,
+      "drain_tail_s": 976.8599372793938,
+      "offered_load": {
+        "decode_tokens_per_s": 222.2658469945355,
+        "prefill_tokens_per_s_after_prefix": 6478.3401639344265,
+        "prefill_tokens_per_s_raw": 14510.624316939891,
+        "requests_per_s": 0.46939890710382515
+      },
+      "prefix_cache_hit_ratio": 0.5536777047530368,
+      "requests": 1718,
+      "rho": 0.0016,
+      "subcritical_gate": false,
+      "topology": "tp2_mns16",
+      "tpot_ms": {
+        "count": 1710,
+        "max": 1075.3968454229532,
+        "mean": 82.12803809595945,
+        "min": 5.044100516215622,
+        "p50": 55.434284109098826,
+        "p90": 148.45522338371572,
+        "p95": 240.8300400647424,
+        "p99": 469.60790088235683
+      },
+      "trace_sha256": "d785665c86a562c9d9a7492ad710e6393bb883bf8c3ad552a34ff18f22925043",
+      "ttft_ms": {
+        "count": 1718,
+        "max": 899333.5996373689,
+        "mean": 363823.5998174686,
+        "min": 95.16785391630833,
+        "p50": 333469.8464378732,
+        "p90": 786932.9549504423,
+        "p95": 836208.7308378813,
+        "p99": 879481.7276833938
+      },
+      "waiting_ms": {
+        "count": 1718,
+        "max": 898008.7571403392,
+        "mean": 362060.3850241827,
+        "min": 0.0,
+        "p50": 332965.98011225695,
+        "p90": 785791.1926569697,
+        "p95": 832975.7759936375,
+        "p99": 872501.6632742453
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {
+          "1": 82578,
+          "10": 992,
+          "11": 686,
+          "12": 564,
+          "13": 388,
+          "14": 207,
+          "15": 120,
+          "16": 556,
+          "2": 76732,
+          "3": 49252,
+          "4": 30080,
+          "5": 19950,
+          "6": 9869,
+          "7": 6575,
+          "8": 3735,
+          "9": 1733
+        },
+        "max": 16,
+        "share_gt_1": 0.7092497984275589,
+        "share_gt_4": 0.15976156356837795,
+        "stages": 284017
+      },
+      "drain_fraction": 0.0057571275011030374,
+      "drain_tail_s": 21.071086654037117,
+      "offered_load": {
+        "decode_tokens_per_s": 222.2658469945355,
+        "prefill_tokens_per_s_after_prefix": 6066.187158469946,
+        "prefill_tokens_per_s_raw": 14510.624316939891,
+        "requests_per_s": 0.46939890710382515
+      },
+      "prefix_cache_hit_ratio": 0.5820880455385098,
+      "requests": 1718,
+      "rho": 0.0016,
+      "subcritical_gate": true,
+      "topology": "tp4_mns16",
+      "tpot_ms": {
+        "count": 1710,
+        "max": 779.2730316123988,
+        "mean": 31.087273162821237,
+        "min": 4.6427767165369005,
+        "p50": 9.443696854706767,
+        "p90": 67.85056779351899,
+        "p95": 145.53426807885694,
+        "p99": 346.8968267376913
+      },
+      "trace_sha256": "d785665c86a562c9d9a7492ad710e6393bb883bf8c3ad552a34ff18f22925043",
+      "ttft_ms": {
+        "count": 1718,
+        "max": 24498.655361823694,
+        "mean": 2344.154809416248,
+        "min": 72.15684780067022,
+        "p50": 646.7976478797937,
+        "p90": 7657.698621058489,
+        "p95": 10053.03089890951,
+        "p99": 16706.358496869783
+      },
+      "waiting_ms": {
+        "count": 1718,
+        "max": 20607.142196811765,
+        "mean": 1395.7184472333554,
+        "min": 0.0,
+        "p50": 5.521385635063325,
+        "p90": 5253.222863654173,
+        "p95": 8173.9598212075725,
+        "p99": 14808.95660872061
+      }
+    }
+  ],
+  "schema": "frontier-code-trace-calibration-summary-v1",
+  "subcritical_rule": "drain tail <= 10% of the 3660s arrival window",
+  "workload_mode": "prefill_decode"
+}

runs/frontier-code-trace-v0/results/canary-analysis-tp2-v1.json

@@ -0,0 +1,319 @@
+{
+  "drain": {
+    "interpretation": "Report the max-completion request explicitly; a response that arrived well before the cutoff can create a long drain tail without implying queue accumulation.",
+    "real_per_trial": [
+      {
+        "last_arrival_s": 597.0069999999832,
+        "last_completion_s": 714.3421455998905,
+        "tail_after_last_arrival_s": 117.33514559990726,
+        "tail_driver": {
+          "admission_lag_ms": 12.391218915581703,
+          "arrival_before_cutoff_s": 62.03099999995902,
+          "arrival_s": 534.9760000000242,
+          "e2e_ms": 179353.7543809507,
+          "input_tokens": 61976,
+          "output_tokens": 21361,
+          "request_index": 40
+        }
+      }
+    ],
+    "sim": {
+      "last_arrival_s": 597.007,
+      "last_completion_s": 713.4565521329489,
+      "tail_after_last_arrival_s": 116.44955213294895,
+      "tail_driver": {
+        "arrival_before_cutoff_s": 62.03099999999995,
+        "arrival_s": 534.976,
+        "e2e_ms": 178480.5521329489,
+        "input_tokens": 61976,
+        "output_tokens": 21361,
+        "request_index": 40,
+        "waiting_ms": 0.0
+      }
+    }
+  },
+  "input": {
+    "frontier_csv_sha256": "624bc983d3c9fba58cbfa3bb5126304b99b19883da4c78f63cd8622b856dbc1f",
+    "manifest": "runs/frontier-code-trace-v0/results/canary-input-r0p0004-v1/manifest.json",
+    "paired_row_vector_sha256": "72b09903851f5684838da23498a068efc409e6481d1e84efeeefddd8e431c3e7"
+  },
+  "latency_ms": {
+    "e2e": {
+      "real": {
+        "count": 53,
+        "max": 179353.7543809507,
+        "mean": 15936.508002247274,
+        "p50": 3162.7415760885924,
+        "p90": 31124.16882226244,
+        "p95": 62989.56617997021,
+        "p99": 165236.24419869846
+      },
+      "real_per_trial": [
+        {
+          "count": 53,
+          "max": 179353.7543809507,
+          "mean": 15936.508002247274,
+          "p50": 3162.7415760885924,
+          "p90": 31124.16882226244,
+          "p95": 62989.56617997021,
+          "p99": 165236.24419869846
+        }
+      ],
+      "relative_bias_percent": {
+        "mean": -8.269543527339733,
+        "p50": -8.321761801888824,
+        "p90": -18.368943324171262,
+        "p95": -23.79834859601531,
+        "p99": -2.75020118151507
+      },
+      "sim": {
+        "count": 53,
+        "max": 178480.5521329489,
+        "mean": 14618.631536263456,
+        "p50": 2899.5457557171953,
+        "p90": 25406.98789118167,
+        "p95": 47999.08964134313,
+        "p99": 160691.91505845473
+      }
+    },
+    "tpot": {
+      "real": {
+        "count": 53,
+        "max": 641.5267530052612,
+        "mean": 27.41422879865018,
+        "p50": 8.106854443332919,
+        "p90": 23.880988038291846,
+        "p95": 92.03784805296183,
+        "p99": 397.1900480028069
+      },
+      "real_per_trial": [
+        {
+          "count": 53,
+          "max": 641.5267530052612,
+          "mean": 27.41422879865018,
+          "p50": 8.106854443332919,
+          "p90": 23.880988038291846,
+          "p95": 92.03784805296183,
+          "p99": 397.1900480028069
+        }
+      ],
+      "relative_bias_percent": {
+        "mean": -9.878641617364694,
+        "p50": -2.3180300975258694,
+        "p90": 26.54184469239794,
+        "p95": -24.72214442719661,
+        "p99": -13.722225303935645
+      },
+      "sim": {
+        "count": 53,
+        "max": 556.599034356792,
+        "mean": 24.706075383467144,
+        "p50": 7.918935117373849,
+        "p90": 30.219442794425397,
+        "p95": 69.28411832962485,
+        "p99": 342.6867347310516
+      }
+    },
+    "ttft": {
+      "real": {
+        "count": 53,
+        "max": 23546.024559997022,
+        "mean": 3438.4982450513767,
+        "p50": 918.7854011543095,
+        "p90": 10432.362293498598,
+        "p95": 13642.840061988678,
+        "p99": 22228.129854062565
+      },
+      "real_per_trial": [
+        {
+          "count": 53,
+          "max": 23546.024559997022,
+          "mean": 3438.4982450513767,
+          "p50": 918.7854011543095,
+          "p90": 10432.362293498598,
+          "p95": 13642.840061988678,
+          "p99": 22228.129854062565
+        }
+      ],
+      "relative_bias_percent": {
+        "mean": -21.116133763518285,
+        "p50": 13.780393777591794,
+        "p90": -32.09252788469729,
+        "p95": -13.833140262007847,
+        "p99": -14.006169043239403
+      },
+      "sim": {
+        "count": 53,
+        "max": 19564.258977976864,
+        "mean": 2712.4203561700992,
+        "p50": 1045.3976474043998,
+        "p90": 7084.353515424915,
+        "p95": 11755.606860492386,
+        "p99": 19114.8204115518
+      }
+    }
+  },
+  "prefix_cache": {
+    "real_hit_ratio_mean": 0.2399084848137731,
+    "real_per_trial": [
+      {
+        "hit_ratio": 0.2399084848137731,
+        "hit_tokens": 309968.0,
+        "query_tokens": 1292026.0
+      }
+    ],
+    "sim": {
+      "block_size_tokens": 16,
+      "hit_ratio": 0.23997274866840085,
+      "mean_cached_prefill_tokens": 5848.452830188679,
+      "requests": 53,
+      "requests_with_hits": 9,
+      "total_cached_prefill_tokens": 309968,
+      "total_hit_blocks": 19373,
+      "total_query_blocks": 80730
+    }
+  },
+  "real_artifacts": [
+    {
+      "result_sha256": "54ebf30bb66fd845d0d6bbdb15674784989070ec12aa8b785f1dd19886707269",
+      "root": "runs/frontier-code-trace-v0/results/real-canary/r0p0004-tp2-t1-v3"
+    }
+  ],
+  "real_decode_batch_per_trial": [
+    {
+      "histogram": {
+        "1": 53281,
+        "2": 5701,
+        "3": 2919,
+        "4": 476,
+        "5": 110,
+        "6": 62
+      },
+      "max": 6,
+      "mixed_steps_excluded": 152,
+      "share_gt_1": 0.14817183328270636,
+      "steps": 62549
+    }
+  ],
+  "requests_per_trial": 53,
+  "schema": "frontier-code-trace-canary-analysis-v1",
+  "sim_decode_batch": {
+    "histogram": {
+      "1": 53233,
+      "2": 6466,
+      "3": 2626,
+      "4": 395,
+      "5": 103,
+      "6": 29
+    },
+    "max": 6,
+    "share_gt_1": 0.1530420670782155,
+    "share_gt_4": 0.0021001718322408197,
+    "stages": 62852
+  },
+  "slo": {
+    "definition": {
+      "target_pass_rate": 0.95,
+      "tpot_ms": 150,
+      "ttft_ms": "1000 + 1000 * input_tokens / 8000"
+    },
+    "feasibility_flip": false,
+    "real_per_trial": [
+      {
+        "feasible": false,
+        "pass_rate": 0.7735849056603774,
+        "passed": 41
+      }
+    ],
+    "sim": {
+      "feasible": false,
+      "pass_rate": 0.8679245283018868,
+      "passed": 46
+    }
+  },
+  "topology": "tp2_mns16",
+  "tpot_by_min_output_tokens": {
+    "2": {
+      "real": {
+        "count": 53,
+        "max": 641.5267530052612,
+        "mean": 27.41422879865018,
+        "p50": 8.106854443332919,
+        "p90": 23.880988038291846,
+        "p95": 92.03784805296183,
+        "p99": 397.1900480028069
+      },
+      "relative_bias_percent": {
+        "mean": -9.878641617364694,
+        "p50": -2.3180300975258694,
+        "p90": 26.54184469239794,
+        "p95": -24.72214442719661,
+        "p99": -13.722225303935645
+      },
+      "sim": {
+        "count": 53,
+        "max": 556.599034356792,
+        "mean": 24.706075383467144,
+        "p50": 7.918935117373849,
+        "p90": 30.219442794425397,
+        "p95": 69.28411832962485,
+        "p99": 342.6867347310516
+      }
+    },
+    "32": {
+      "real": {
+        "count": 45,
+        "max": 171.6484741543905,
+        "mean": 16.593114914019644,
+        "p50": 8.106854443332919,
+        "p90": 19.44677900381304,
+        "p95": 66.47742398097495,
+        "p99": 148.17240555253153
+      },
+      "relative_bias_percent": {
+        "mean": -9.940572863733157,
+        "p50": -2.3180300975258694,
+        "p90": 10.853794302071227,
+        "p95": -22.75922601837412,
+        "p99": -18.755468241526987
+      },
+      "sim": {
+        "count": 45,
+        "max": 145.22922738421678,
+        "mean": 14.943664235628548,
+        "p50": 7.918935117373849,
+        "p90": 21.557492395265285,
+        "p95": 51.34767680595202,
+        "p99": 120.38197708641991
+      }
+    },
+    "8": {
+      "real": {
+        "count": 52,
+        "max": 171.6484741543905,
+        "mean": 15.604372563907663,
+        "p50": 8.031664372446695,
+        "p90": 21.733123898935485,
+        "p95": 52.378594044159485,
+        "p99": 144.4375764567812
+      },
+      "relative_bias_percent": {
+        "mean": -7.222385426340451,
+        "p50": -1.403560331275464,
+        "p90": 10.95444148849469,
+        "p95": -10.968821104857895,
+        "p99": -19.391471177881424
+      },
+      "sim": {
+        "count": 52,
+        "max": 145.22922738421678,
+        "mean": 14.477364633980129,
+        "p50": 7.918935117373849,
+        "p90": 24.11386624006643,
+        "p95": 46.63327976621588,
+        "p99": 116.42900544813402
+      }
+    }
+  },
+  "trials": 1
+}

runs/frontier-code-trace-v0/results/canary-analysis-tp4-v2.json

@@ -0,0 +1,484 @@
+{
+  "drain": {
+    "interpretation": "Report the max-completion request explicitly; a response that arrived well before the cutoff can create a long drain tail without implying queue accumulation.",
+    "real_per_trial": [
+      {
+        "last_arrival_s": 597.0069999999832,
+        "last_completion_s": 689.6167901020963,
+        "tail_after_last_arrival_s": 92.60979010211304,
+        "tail_driver": {
+          "admission_lag_ms": 11.249033967033029,
+          "arrival_before_cutoff_s": 62.03099999995902,
+          "arrival_s": 534.9760000000242,
+          "e2e_ms": 154629.54106810503,
+          "input_tokens": 61976,
+          "output_tokens": 21361,
+          "request_index": 40
+        }
+      },
+      {
+        "last_arrival_s": 597.0069999999832,
+        "last_completion_s": 689.344073610846,
+        "tail_after_last_arrival_s": 92.33707361086272,
+        "tail_driver": {
+          "admission_lag_ms": 12.91605201549828,
+          "arrival_before_cutoff_s": 62.03099999995902,
+          "arrival_s": 534.9760000000242,
+          "e2e_ms": 154355.15755880624,
+          "input_tokens": 61976,
+          "output_tokens": 21361,
+          "request_index": 40
+        }
+      }
+    ],
+    "sim": {
+      "last_arrival_s": 597.007,
+      "last_completion_s": 688.377327361797,
+      "tail_after_last_arrival_s": 91.37032736179708,
+      "tail_driver": {
+        "arrival_before_cutoff_s": 62.03099999999995,
+        "arrival_s": 534.976,
+        "e2e_ms": 153401.32736179704,
+        "input_tokens": 61976,
+        "output_tokens": 21361,
+        "request_index": 40,
+        "waiting_ms": 0.0
+      }
+    }
+  },
+  "input": {
+    "frontier_csv_sha256": "624bc983d3c9fba58cbfa3bb5126304b99b19883da4c78f63cd8622b856dbc1f",
+    "manifest": "runs/frontier-code-trace-v0/results/canary-input-r0p0004-v1/manifest.json",
+    "paired_row_vector_sha256": "72b09903851f5684838da23498a068efc409e6481d1e84efeeefddd8e431c3e7"
+  },
+  "latency_ms": {
+    "e2e": {
+      "real": {
+        "count": 106,
+        "max": 154629.54106810503,
+        "mean": 10734.692601765997,
+        "p50": 1698.6070264829323,
+        "p90": 16955.372447962873,
+        "p95": 40098.011117079295,
+        "p99": 152916.82229990847
+      },
+      "real_per_trial": [
+        {
+          "count": 53,
+          "max": 154629.54106810503,
+          "mean": 10752.865282098219,
+          "p50": 1699.7638561297208,
+          "p90": 16719.75081679412,
+          "p95": 32787.88763261397,
+          "p99": 139528.17495073183
+        },
+        {
+          "count": 53,
+          "max": 154355.15755880624,
+          "mean": 10716.519921433777,
+          "p50": 1697.8129579219967,
+          "p90": 16614.017988741405,
+          "p95": 32681.116299051762,
+          "p99": 139096.4567810855
+        }
+      ],
+      "real_trial_statistic_mean": {
+        "mean": 10734.692601765997,
+        "p50": 1698.7884070258588,
+        "p90": 16666.884402767762,
+        "p95": 32734.501965832867,
+        "p99": 139312.31586590866
+      },
+      "relative_bias_percent": {
+        "mean": 2.2014558826783013,
+        "p50": 6.0982380145415895,
+        "p90": 11.04630840126803,
+        "p95": 1.3549410756255433,
+        "p99": -1.3257901787372948
+      },
+      "sim": {
+        "count": 53,
+        "max": 153401.32736179704,
+        "mean": 10971.012123535007,
+        "p50": 1802.3845674497352,
+        "p90": 18507.95985478033,
+        "p95": 33178.03517886939,
+        "p99": 137465.32686438697
+      }
+    },
+    "tpot": {
+      "real": {
+        "count": 106,
+        "max": 212.7724236343056,
+        "mean": 10.903347021010111,
+        "p50": 6.172303046257078,
+        "p90": 8.493485802941574,
+        "p95": 9.056036081054664,
+        "p99": 203.35920991970733
+      },
+      "real_per_trial": [
+        {
+          "count": 53,
+          "max": 212.7724236343056,
+          "mean": 10.931995250979348,
+          "p50": 6.17668606558597,
+          "p90": 8.407062460850472,
+          "p95": 8.930405185071265,
+          "p99": 137.66455429146993
+        },
+        {
+          "count": 53,
+          "max": 210.46578868602714,
+          "mean": 10.874698791040872,
+          "p50": 6.167920026928186,
+          "p90": 8.391951717689471,
+          "p95": 8.880314963829257,
+          "p99": 136.53404073389595
+        }
+      ],
+      "real_trial_statistic_mean": {
+        "mean": 10.90334702101011,
+        "p50": 6.172303046257078,
+        "p90": 8.39950708926997,
+        "p95": 8.905360074450261,
+        "p99": 137.09929751268294
+      },
+      "relative_bias_percent": {
+        "mean": -29.893478835305263,
+        "p50": 8.372972857395876,
+        "p90": -4.849833918394976,
+        "p95": 9.343177342624926,
+        "p99": -71.54426929141353
+      },
+      "sim": {
+        "count": 53,
+        "max": 68.73522813087318,
+        "mean": 7.643957286944565,
+        "p50": 6.6891083049964015,
+        "p90": 7.992144945476565,
+        "p95": 9.737403659205464,
+        "p99": 39.01260690357286
+      }
+    },
+    "ttft": {
+      "real": {
+        "count": 106,
+        "max": 12213.839336996898,
+        "mean": 1499.2377276352238,
+        "p50": 523.7399585312232,
+        "p90": 3843.0782224750146,
+        "p95": 7372.6646098075435,
+        "p99": 12137.850578338843
+      },
+      "real_per_trial": [
+        {
+          "count": 53,
+          "max": 12213.839336996898,
+          "mean": 1501.9183055506492,
+          "p50": 524.1345730610192,
+          "p90": 3807.3078704066584,
+          "p95": 6591.913985600691,
+          "p99": 11615.677419919517
+        },
+        {
+          "count": 53,
+          "max": 12194.393874146044,
+          "mean": 1496.5571497197984,
+          "p50": 523.3453440014273,
+          "p90": 3795.468537742273,
+          "p95": 6574.768227571618,
+          "p99": 11596.839222898703
+        }
+      ],
+      "real_trial_statistic_mean": {
+        "mean": 1499.2377276352238,
+        "p50": 523.7399585312232,
+        "p90": 3801.3882040744656,
+        "p95": 6583.341106586155,
+        "p99": 11606.25832140911
+      },
+      "relative_bias_percent": {
+        "mean": -8.33138601964073,
+        "p50": -4.13255547614917,
+        "p90": -8.820129670118508,
+        "p95": -13.08726396978494,
+        "p99": -6.311340671626045
+      },
+      "sim": {
+        "count": 53,
+        "max": 11454.943556960472,
+        "mean": 1374.3304451938434,
+        "p50": 502.0961141941598,
+        "p90": 3466.1008352105086,
+        "p95": 5721.761877935864,
+        "p99": 10873.747819516035
+      }
+    }
+  },
+  "prefix_cache": {
+    "real_hit_ratio_mean": 0.2399084848137731,
+    "real_per_trial": [
+      {
+        "hit_ratio": 0.2399084848137731,
+        "hit_tokens": 309968.0,
+        "query_tokens": 1292026.0
+      },
+      {
+        "hit_ratio": 0.2399084848137731,
+        "hit_tokens": 309968.0,
+        "query_tokens": 1292026.0
+      }
+    ],
+    "sim": {
+      "block_size_tokens": 16,
+      "hit_ratio": 0.23997274866840085,
+      "mean_cached_prefill_tokens": 5848.452830188679,
+      "requests": 53,
+      "requests_with_hits": 9,
+      "total_cached_prefill_tokens": 309968,
+      "total_hit_blocks": 19373,
+      "total_query_blocks": 80730
+    }
+  },
+  "real_artifacts": [
+    {
+      "result_sha256": "c1105d6ba182be86fcd85f4eaf6889d1e788d48f923779b742ab95b4d3bd46ca",
+      "root": "runs/frontier-code-trace-v0/results/real-canary/r0p0004-tp4-t1-v2"
+    },
+    {
+      "result_sha256": "8e9092ac94b9d84d13aca41bd41d33db67d9917c8cedc8b61854aa2963a124ce",
+      "root": "runs/frontier-code-trace-v0/results/real-canary/r0p0004-tp4-t2-v3"
+    }
+  ],
+  "real_decode_batch_per_trial": [
+    {
+      "histogram": {
+        "1": 54699,
+        "2": 9377,
+        "3": 867,
+        "4": 77
+      },
+      "max": 4,
+      "mixed_steps_excluded": 155,
+      "share_gt_1": 0.15873577360812058,
+      "steps": 65020
+    },
+    {
+      "histogram": {
+        "1": 54893,
+        "2": 9265,
+        "3": 877,
+        "4": 77
+      },
+      "max": 4,
+      "mixed_steps_excluded": 155,
+      "share_gt_1": 0.15694495638284803,
+      "steps": 65112
+    }
+  ],
+  "requests_per_trial": 53,
+  "schema": "frontier-code-trace-canary-analysis-v1",
+  "sim_decode_batch": {
+    "histogram": {
+      "1": 54192,
+      "2": 9673,
+      "3": 840,
+      "4": 77
+    },
+    "max": 4,
+    "share_gt_1": 0.1634713346299898,
+    "share_gt_4": 0.0,
+    "stages": 64782
+  },
+  "slo": {
+    "definition": {
+      "target_pass_rate": 0.95,
+      "tpot_ms": 150,
+      "ttft_ms": "1000 + 1000 * input_tokens / 8000"
+    },
+    "feasibility_flip": true,
+    "real_per_trial": [
+      {
+        "feasible": false,
+        "pass_rate": 0.9433962264150944,
+        "passed": 50
+      },
+      {
+        "feasible": false,
+        "pass_rate": 0.9433962264150944,
+        "passed": 50
+      }
+    ],
+    "sim": {
+      "feasible": true,
+      "pass_rate": 0.9811320754716981,
+      "passed": 52
+    }
+  },
+  "topology": "tp4_mns16",
+  "tpot_by_min_output_tokens": {
+    "2": {
+      "real": {
+        "count": 106,
+        "max": 212.7724236343056,
+        "mean": 10.903347021010111,
+        "p50": 6.172303046257078,
+        "p90": 8.493485802941574,
+        "p95": 9.056036081054664,
+        "p99": 203.35920991970733
+      },
+      "real_per_trial": [
+        {
+          "count": 53,
+          "max": 212.7724236343056,
+          "mean": 10.931995250979348,
+          "p50": 6.17668606558597,
+          "p90": 8.407062460850472,
+          "p95": 8.930405185071265,
+          "p99": 137.66455429146993
+        },
+        {
+          "count": 53,
+          "max": 210.46578868602714,
+          "mean": 10.874698791040872,
+          "p50": 6.167920026928186,
+          "p90": 8.391951717689471,
+          "p95": 8.880314963829257,
+          "p99": 136.53404073389595
+        }
+      ],
+      "real_trial_statistic_mean": {
+        "mean": 10.90334702101011,
+        "p50": 6.172303046257078,
+        "p90": 8.39950708926997,
+        "p95": 8.905360074450261,
+        "p99": 137.09929751268294
+      },
+      "relative_bias_percent": {
+        "mean": -29.893478835305263,
+        "p50": 8.372972857395876,
+        "p90": -4.849833918394976,
+        "p95": 9.343177342624926,
+        "p99": -71.54426929141353
+      },
+      "sim": {
+        "count": 53,
+        "max": 68.73522813087318,
+        "mean": 7.643957286944565,
+        "p50": 6.6891083049964015,
+        "p90": 7.992144945476565,
+        "p95": 9.737403659205464,
+        "p99": 39.01260690357286
+      }
+    },
+    "32": {
+      "real": {
+        "count": 90,
+        "max": 68.33421335962247,
+        "mean": 7.270237487352443,
+        "p50": 6.167292870073156,
+        "p90": 8.490015433923487,
+        "p95": 8.695471711641535,
+        "p99": 68.2942852512835
+      },
+      "real_per_trial": [
+        {
+          "count": 45,
+          "max": 68.33421335962247,
+          "mean": 7.2794157345844095,
+          "p50": 6.17668606558597,
+          "p90": 8.316301157486755,
+          "p95": 8.667796532745909,
+          "p99": 42.34591449575732
+        },
+        {
+          "count": 45,
+          "max": 68.28935031654497,
+          "mean": 7.261059240120478,
+          "p50": 6.157899674560342,
+          "p90": 8.294755593709974,
+          "p95": 8.647785231561759,
+          "p99": 42.27831274721521
+        }
+      ],
+      "real_trial_statistic_mean": {
+        "mean": 7.270237487352444,
+        "p50": 6.167292870073156,
+        "p90": 8.305528375598364,
+        "p95": 8.657790882153833,
+        "p99": 42.31211362148626
+      },
+      "relative_bias_percent": {
+        "mean": 8.827198903080898,
+        "p50": 10.602910786790718,
+        "p90": 3.6523081450671953,
+        "p95": 13.443742144167812,
+        "p99": 3.009077380352242
+      },
+      "sim": {
+        "count": 45,
+        "max": 68.73522813087318,
+        "mean": 7.911995811087395,
+        "p50": 6.821205431047118,
+        "p90": 8.60887186495121,
+        "p95": 9.821721963731866,
+        "p99": 43.585317861619345
+      }
+    },
+    "8": {
+      "real": {
+        "count": 104,
+        "max": 68.33421335962247,
+        "mean": 7.043428576026337,
+        "p50": 6.162909850744264,
+        "p90": 8.031762239708375,
+        "p95": 8.660612540767117,
+        "p99": 66.51876673984641
+      },
+      "real_per_trial": [
+        {
+          "count": 52,
+          "max": 68.33421335962247,
+          "mean": 7.050448551299998,
+          "p50": 6.167180339419747,
+          "p90": 8.00607090381441,
+          "p95": 8.604306061843479,
+          "p99": 38.21141240377873
+        },
+        {
+          "count": 52,
+          "max": 68.28935031654497,
+          "mean": 7.036408600752676,
+          "p50": 6.162909850744264,
+          "p90": 7.971146045984491,
+          "p95": 8.582739223220587,
+          "p99": 38.14019313391272
+        }
+      ],
+      "real_trial_statistic_mean": {
+        "mean": 7.043428576026336,
+        "p50": 6.1650450950820055,
+        "p90": 7.988608474899451,
+        "p95": 8.593522642532033,
+        "p99": 38.17580276884573
+      },
+      "relative_bias_percent": {
+        "mean": 8.736391650572934,
+        "p50": 8.20225561474644,
+        "p90": 0.591743534737434,
+        "p95": 13.433615674969445,
+        "p99": 3.6892295703930555
+      },
+      "sim": {
+        "count": 52,
+        "max": 68.73522813087318,
+        "mean": 7.658770082056369,
+        "p50": 6.670717852545019,
+        "p90": 8.035880549065155,
+        "p95": 9.747943447271265,
+        "p99": 39.584195773328915
+      }
+    }
+  },
+  "trials": 2
+}

runs/frontier-code-trace-v0/results/max-model-len-smoke-tp4-v2/provenance/gpus.after.csv

@@ -0,0 +1,8 @@
+0, NVIDIA H20, 0, 0
+1, NVIDIA H20, 0, 0
+2, NVIDIA H20, 0, 0
+3, NVIDIA H20, 0, 0
+4, NVIDIA H20, 0, 0
+5, NVIDIA H20, 0, 0
+6, NVIDIA H20, 0, 0
+7, NVIDIA H20, 0, 0

runs/frontier-code-trace-v0/results/max-model-len-smoke-tp4-v2/provenance/gpus.before.csv

@@ -0,0 +1,8 @@
+0, NVIDIA H20, 0, 0, 0
+1, NVIDIA H20, 0, 0, 0
+2, NVIDIA H20, 0, 0, 0
+3, NVIDIA H20, 0, 0, 0
+4, NVIDIA H20, 0, 0, 0
+5, NVIDIA H20, 0, 0, 0
+6, NVIDIA H20, 0, 0, 0
+7, NVIDIA H20, 0, 0, 0

runs/frontier-code-trace-v0/results/max-model-len-smoke-tp4-v2/provenance/sources.sha256

@@ -0,0 +1,3 @@
+9857f3ee2565e0ae5bf414c769ea7404295eb7b15fc4a4e8c0f85726312e27a5  runs/frontier-code-trace-v0/run_max_model_len_smoke.sh
+0b6a6d1d8c71f57b8e11b4eb89a9887e182a327223206577086605930da3c028  /home/admin/cpfs/wjh/aituner/aituner-code-smoke-da480a8/runs/frontier-s3-real-v0/qwen30_prefill_client.py
+2850ddb3bf7aecad20b611e2d44f3077fc8193f4827c93beddd4c02ad63c2297  /home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B/config.json

runs/frontier-code-trace-v0/results/max-model-len-smoke-tp4-v2/results/code-candidate-max.json

@@ -0,0 +1,47 @@
+{
+  "requests": [
+    {
+      "admission_lag_ms": 0.09795790538191795,
+      "admitted_s": 9.795790538191795e-05,
+      "completion_tokens": 242,
+      "e2e_ms": 5778.945273021236,
+      "prompt_tokens": 136774,
+      "request_index": 0,
+      "scheduled_s": 0.0,
+      "slo_pass": true,
+      "streamed_token_count": 242,
+      "success": true,
+      "tpot_ms": 8.126009058562799,
+      "ttft_ms": 3820.549983996898
+    }
+  ],
+  "schema": "qwen30-fixed-rate-anchor-v2",
+  "summary": {
+    "admission_lag_max_ms": 0.09795790538191795,
+    "completed": 1,
+    "failed": 0,
+    "slo": {
+      "feasible": true,
+      "pass_rate": 1.0,
+      "passed": 1,
+      "tpot_threshold_ms": 150.0,
+      "ttft_threshold_ms": 18096.75
+    },
+    "tpot_max_ms": 8.126009058562799,
+    "tpot_p50_ms": 8.126009058562799,
+    "tpot_p95_ms": 8.126009058562799,
+    "ttft_max_ms": 3820.549983996898,
+    "ttft_p50_ms": 3820.549983996898,
+    "ttft_p95_ms": 3820.549983996898
+  },
+  "workload": {
+    "arrival": "open_loop_uniform",
+    "input_tokens": 136774,
+    "last_scheduled_arrival_s": 0.0,
+    "offered_request_rate": 1.0,
+    "output_tokens": 242,
+    "prefix_caching": false,
+    "prompt_vector_sha256": "40852e3d159377ae557b58933b077bee1e565bc51a4b7d4e3ea0070709e63999",
+    "request_count": 1
+  }
+}

runs/frontier-code-trace-v0/results/max-model-len-smoke-tp4-v2/results/code-candidate-p99.json

@@ -0,0 +1,47 @@
+{
+  "requests": [
+    {
+      "admission_lag_ms": 0.10300683788955212,
+      "admitted_s": 0.00010300683788955212,
+      "completion_tokens": 68,
+      "e2e_ms": 13182.361921994016,
+      "prompt_tokens": 119702,
+      "request_index": 0,
+      "scheduled_s": 0.0,
+      "slo_pass": true,
+      "streamed_token_count": 68,
+      "success": true,
+      "tpot_ms": 7.21338267243509,
+      "ttft_ms": 12699.044595938176
+    }
+  ],
+  "schema": "qwen30-fixed-rate-anchor-v2",
+  "summary": {
+    "admission_lag_max_ms": 0.10300683788955212,
+    "completed": 1,
+    "failed": 0,
+    "slo": {
+      "feasible": true,
+      "pass_rate": 1.0,
+      "passed": 1,
+      "tpot_threshold_ms": 150.0,
+      "ttft_threshold_ms": 15962.75
+    },
+    "tpot_max_ms": 7.21338267243509,
+    "tpot_p50_ms": 7.21338267243509,
+    "tpot_p95_ms": 7.21338267243509,
+    "ttft_max_ms": 12699.044595938176,
+    "ttft_p50_ms": 12699.044595938176,
+    "ttft_p95_ms": 12699.044595938176
+  },
+  "workload": {
+    "arrival": "open_loop_uniform",
+    "input_tokens": 119702,
+    "last_scheduled_arrival_s": 0.0,
+    "offered_request_rate": 1.0,
+    "output_tokens": 68,
+    "prefix_caching": false,
+    "prompt_vector_sha256": "aaa2e7077defd41a1abb6227d9d22ec928a5d56e18fb3135783a6044e8365af7",
+    "request_count": 1
+  }
+}

runs/frontier-code-trace-v0/results/max-model-len-smoke-tp4-v2/results/code-p50.json

@@ -0,0 +1,47 @@
+{
+  "requests": [
+    {
+      "admission_lag_ms": 0.08747284300625324,
+      "admitted_s": 8.747284300625324e-05,
+      "completion_tokens": 78,
+      "e2e_ms": 2135.5441550258547,
+      "prompt_tokens": 20051,
+      "request_index": 0,
+      "scheduled_s": 0.0,
+      "slo_pass": true,
+      "streamed_token_count": 78,
+      "success": true,
+      "tpot_ms": 16.65233903830605,
+      "ttft_ms": 853.2876779790968
+    }
+  ],
+  "schema": "qwen30-fixed-rate-anchor-v2",
+  "summary": {
+    "admission_lag_max_ms": 0.08747284300625324,
+    "completed": 1,
+    "failed": 0,
+    "slo": {
+      "feasible": true,
+      "pass_rate": 1.0,
+      "passed": 1,
+      "tpot_threshold_ms": 150.0,
+      "ttft_threshold_ms": 3506.375
+    },
+    "tpot_max_ms": 16.65233903830605,
+    "tpot_p50_ms": 16.65233903830605,
+    "tpot_p95_ms": 16.65233903830605,
+    "ttft_max_ms": 853.2876779790968,
+    "ttft_p50_ms": 853.2876779790968,
+    "ttft_p95_ms": 853.2876779790968
+  },
+  "workload": {
+    "arrival": "open_loop_uniform",
+    "input_tokens": 20051,
+    "last_scheduled_arrival_s": 0.0,
+    "offered_request_rate": 1.0,
+    "output_tokens": 78,
+    "prefix_caching": false,
+    "prompt_vector_sha256": "a814b02092178b7e949e4d14d15aa271c9e93cead6552d91413d8f9abf2c9287",
+    "request_count": 1
+  }
+}

runs/frontier-code-trace-v0/results/max-model-len-smoke-tp4-v2/results/models.json

@@ -0,0 +1 @@
+{"object":"list","data":[{"id":"qwen3-30b-code-maxlen-smoke","object":"model","created":1784827167,"owned_by":"vllm","root":"/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B","parent":null,"max_model_len":147456,"permission":[{"id":"modelperm-971c6c18ac176f4d","object":"model_permission","created":1784827167,"allow_create_engine":false,"allow_sampling":true,"allow_logprobs":true,"allow_search_indices":false,"allow_view":true,"allow_fine_tuning":false,"organization":"*","group":null,"is_blocking":false}]}]}

runs/frontier-code-trace-v0/results/max-model-len-smoke-tp4-v2/results/warmup.json

@@ -0,0 +1,47 @@
+{
+  "requests": [
+    {
+      "admission_lag_ms": 0.11776993051171303,
+      "admitted_s": 0.00011776993051171303,
+      "completion_tokens": 1,
+      "e2e_ms": 1389.798145974055,
+      "prompt_tokens": 512,
+      "request_index": 0,
+      "scheduled_s": 0.0,
+      "slo_pass": false,
+      "streamed_token_count": 1,
+      "success": true,
+      "tpot_ms": null,
+      "ttft_ms": 1389.7535749711096
+    }
+  ],
+  "schema": "qwen30-fixed-rate-anchor-v2",
+  "summary": {
+    "admission_lag_max_ms": 0.11776993051171303,
+    "completed": 1,
+    "failed": 0,
+    "slo": {
+      "feasible": false,
+      "pass_rate": 0.0,
+      "passed": 0,
+      "tpot_threshold_ms": null,
+      "ttft_threshold_ms": 1064.0
+    },
+    "tpot_max_ms": null,
+    "tpot_p50_ms": null,
+    "tpot_p95_ms": null,
+    "ttft_max_ms": 1389.7535749711096,
+    "ttft_p50_ms": 1389.7535749711096,
+    "ttft_p95_ms": 1389.7535749711096
+  },
+  "workload": {
+    "arrival": "open_loop_uniform",
+    "input_tokens": 512,
+    "last_scheduled_arrival_s": 0.0,
+    "offered_request_rate": 1.0,
+    "output_tokens": 1,
+    "prefix_caching": false,
+    "prompt_vector_sha256": "11f5ac36e39b39336da6e64e7879dc7fcdedf26e9315bbb267d59ed1fa644939",
+    "request_count": 1
+  }
+}

runs/frontier-code-trace-v0/results/paired-input-manifests/full-r0p0002-v1.json

@@ -0,0 +1,69 @@
+{
+  "block_contract": {
+    "input_length_mismatches": 0,
+    "runtime_identity_collisions": 0,
+    "runtime_to_source_relation_conflicts": 0,
+    "source_hash_count_mismatches": 0,
+    "source_runtime_granularity": "one 512-token source hash to 32 consecutive 16-token identities; final partial source block may have fewer",
+    "source_to_runtime_relation_conflicts": 0,
+    "unique_runtime_identities": 2613467,
+    "unique_runtime_relations": 0,
+    "unique_source_hash_relations": 0
+  },
+  "excluded_over_max_total_tokens": 0,
+  "frontier_csv": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/full-r0p0002-v1/frontier.csv",
+  "frontier_csv_sha256": "188f9e52674ec268d70d763995f2cdf84ccbcbab86420214b127994569324e9b",
+  "mapping": "real prompt tokens: BLAKE2b-128(parent runtime identity, exact 16-token block); missing prompt fallback: source hashes first define deterministic 16-token content blocks, then the same parent-sensitive BLAKE2b-128 runtime identity contract is applied",
+  "max_total_tokens_filter": null,
+  "paired_row_vector_sha256": "6d0ac009dd415a6dab23928afb1eed2947f55d57e3ac6284629a9a6549ce016e",
+  "parent_validation": {
+    "enabled": false,
+    "links_checked": 0,
+    "links_outside_selected_stream": 0,
+    "mean_tail_nonreused_per_link": 0.0,
+    "parent_prefix_common_blocks": 0,
+    "parent_tail_nonreused_blocks": 0,
+    "rule": "contiguous common prefix required; mismatch tolerated only within parent trailing partial 512-token block (<= 33 runtime blocks)"
+  },
+  "prefix_cache_blocks": "complete 16-token blocks only; final partial block excluded",
+  "prompt_contract": {
+    "input_is_pre_remapped": true,
+    "real_prompt_requests": 0,
+    "synthetic_fallback_requests": 232,
+    "tokenizer": null
+  },
+  "prompt_window": null,
+  "prompt_window_sha256": null,
+  "real_requests": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/full-r0p0002-v1/real_requests.jsonl",
+  "real_requests_sha256": "e495a0496fd7183980f2f59f178ac6f5869c4da9deadde45e971f466324674c8",
+  "requests": 232,
+  "rho": 0.0002,
+  "sampling_rule": "keep iff sampling_u <= rho (SimFid convention; rho=1 keeps all rows)",
+  "schema": "frontier-s3-real-remap-v2",
+  "selected_remapped": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/full-r0p0002-v1/selected-remapped.jsonl",
+  "selected_remapped_sha256": "b4b90f2a09bee765365274d5a9d4fb6322761c85b37b9d3a3eee320c268b2c85",
+  "source": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/remap-cache-r0p0032-v2/selected-remapped.jsonl",
+  "source_block_size": 512,
+  "source_blocks": 17691,
+  "source_rows_scanned": 3477,
+  "source_sha256": "f46c7d8ddbd146bafe00fbc173bf3f29039d8b96aa611f1ae4e2a0b9eac189f1",
+  "synthetic_fallback_contract": {
+    "different_16_block_hash_different_tokens": true,
+    "encoding": "injective base-(vocab_size-token_offset), 16 little-endian digits",
+    "runtime_identity_is_parent_sensitive": true,
+    "same_16_block_hash_same_tokens": true,
+    "token_offset": 1024,
+    "vocab_size": 151936
+  },
+  "target_16_blocks": 562555,
+  "target_block_size": 16,
+  "total_decode_tokens": 83052,
+  "total_prefill_tokens": 9002662,
+  "upstream_remap_manifest": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/remap-cache-r0p0032-v2/manifest.json",
+  "upstream_remap_manifest_sha256": "8649b6acf10007e16a6b0eccebe95ee25a33481d6c3c0381ec8caf6dad05f12d",
+  "window": {
+    "duration_s": null,
+    "start_timestamp": null
+  },
+  "workload_mode": "prefill_decode"
+}

runs/frontier-code-trace-v0/results/paired-input-manifests/full-r0p0004-v1.json

@@ -0,0 +1,69 @@
+{
+  "block_contract": {
+    "input_length_mismatches": 0,
+    "runtime_identity_collisions": 0,
+    "runtime_to_source_relation_conflicts": 0,
+    "source_hash_count_mismatches": 0,
+    "source_runtime_granularity": "one 512-token source hash to 32 consecutive 16-token identities; final partial source block may have fewer",
+    "source_to_runtime_relation_conflicts": 0,
+    "unique_runtime_identities": 2613467,
+    "unique_runtime_relations": 0,
+    "unique_source_hash_relations": 0
+  },
+  "excluded_over_max_total_tokens": 0,
+  "frontier_csv": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/full-r0p0004-v1/frontier.csv",
+  "frontier_csv_sha256": "2751c373bb91eccf42b94e1b28245870d6ffa2f91e1b5dabafe2435bc46da274",
+  "mapping": "real prompt tokens: BLAKE2b-128(parent runtime identity, exact 16-token block); missing prompt fallback: source hashes first define deterministic 16-token content blocks, then the same parent-sensitive BLAKE2b-128 runtime identity contract is applied",
+  "max_total_tokens_filter": null,
+  "paired_row_vector_sha256": "2a3e10113bff788484615e3ff90399c0a1ca72427a2591ae38d986d9708019af",
+  "parent_validation": {
+    "enabled": false,
+    "links_checked": 0,
+    "links_outside_selected_stream": 0,
+    "mean_tail_nonreused_per_link": 0.0,
+    "parent_prefix_common_blocks": 0,
+    "parent_tail_nonreused_blocks": 0,
+    "rule": "contiguous common prefix required; mismatch tolerated only within parent trailing partial 512-token block (<= 33 runtime blocks)"
+  },
+  "prefix_cache_blocks": "complete 16-token blocks only; final partial block excluded",
+  "prompt_contract": {
+    "input_is_pre_remapped": true,
+    "real_prompt_requests": 0,
+    "synthetic_fallback_requests": 499,
+    "tokenizer": null
+  },
+  "prompt_window": null,
+  "prompt_window_sha256": null,
+  "real_requests": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/full-r0p0004-v1/real_requests.jsonl",
+  "real_requests_sha256": "f011d6233c12b45ca8054b2cbf79e3241654704d3df4964c1ca39f08aa00b71e",
+  "requests": 499,
+  "rho": 0.0004,
+  "sampling_rule": "keep iff sampling_u <= rho (SimFid convention; rho=1 keeps all rows)",
+  "schema": "frontier-s3-real-remap-v2",
+  "selected_remapped": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/full-r0p0004-v1/selected-remapped.jsonl",
+  "selected_remapped_sha256": "488ff7ed8d641f50f7b850c1ad4f61b2af57dc33474ccbe1e8c1dc475af4d897",
+  "source": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/remap-cache-r0p0032-v2/selected-remapped.jsonl",
+  "source_block_size": 512,
+  "source_blocks": 40421,
+  "source_rows_scanned": 3477,
+  "source_sha256": "f46c7d8ddbd146bafe00fbc173bf3f29039d8b96aa611f1ae4e2a0b9eac189f1",
+  "synthetic_fallback_contract": {
+    "different_16_block_hash_different_tokens": true,
+    "encoding": "injective base-(vocab_size-token_offset), 16 little-endian digits",
+    "runtime_identity_is_parent_sensitive": true,
+    "same_16_block_hash_same_tokens": true,
+    "token_offset": 1024,
+    "vocab_size": 151936
+  },
+  "target_16_blocks": 1285546,
+  "target_block_size": 16,
+  "total_decode_tokens": 330500,
+  "total_prefill_tokens": 20572563,
+  "upstream_remap_manifest": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/remap-cache-r0p0032-v2/manifest.json",
+  "upstream_remap_manifest_sha256": "8649b6acf10007e16a6b0eccebe95ee25a33481d6c3c0381ec8caf6dad05f12d",
+  "window": {
+    "duration_s": null,
+    "start_timestamp": null
+  },
+  "workload_mode": "prefill_decode"
+}

runs/frontier-code-trace-v0/results/paired-input-manifests/full-r0p0008-v1.json

@@ -0,0 +1,69 @@
+{
+  "block_contract": {
+    "input_length_mismatches": 0,
+    "runtime_identity_collisions": 0,
+    "runtime_to_source_relation_conflicts": 0,
+    "source_hash_count_mismatches": 0,
+    "source_runtime_granularity": "one 512-token source hash to 32 consecutive 16-token identities; final partial source block may have fewer",
+    "source_to_runtime_relation_conflicts": 0,
+    "unique_runtime_identities": 2613467,
+    "unique_runtime_relations": 0,
+    "unique_source_hash_relations": 0
+  },
+  "excluded_over_max_total_tokens": 0,
+  "frontier_csv": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/full-r0p0008-v1/frontier.csv",
+  "frontier_csv_sha256": "2788a21c6bb8be6d03fb5151f2c51efa09c0eb5e1f941af60c00a7cc11d41206",
+  "mapping": "real prompt tokens: BLAKE2b-128(parent runtime identity, exact 16-token block); missing prompt fallback: source hashes first define deterministic 16-token content blocks, then the same parent-sensitive BLAKE2b-128 runtime identity contract is applied",
+  "max_total_tokens_filter": null,
+  "paired_row_vector_sha256": "2a0fe9672c0711f7f74847bef400c699167fce8648611fc2ef12cf524b20d47f",
+  "parent_validation": {
+    "enabled": false,
+    "links_checked": 0,
+    "links_outside_selected_stream": 0,
+    "mean_tail_nonreused_per_link": 0.0,
+    "parent_prefix_common_blocks": 0,
+    "parent_tail_nonreused_blocks": 0,
+    "rule": "contiguous common prefix required; mismatch tolerated only within parent trailing partial 512-token block (<= 33 runtime blocks)"
+  },
+  "prefix_cache_blocks": "complete 16-token blocks only; final partial block excluded",
+  "prompt_contract": {
+    "input_is_pre_remapped": true,
+    "real_prompt_requests": 0,
+    "synthetic_fallback_requests": 869,
+    "tokenizer": null
+  },
+  "prompt_window": null,
+  "prompt_window_sha256": null,
+  "real_requests": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/full-r0p0008-v1/real_requests.jsonl",
+  "real_requests_sha256": "aed9d150df986476172ca1d2cce8e9a9f264e24507174265bda9b3e85298cfbe",
+  "requests": 869,
+  "rho": 0.0008,
+  "sampling_rule": "keep iff sampling_u <= rho (SimFid convention; rho=1 keeps all rows)",
+  "schema": "frontier-s3-real-remap-v2",
+  "selected_remapped": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/full-r0p0008-v1/selected-remapped.jsonl",
+  "selected_remapped_sha256": "5ec7b98153cba5549a57e2f15f56f9800f14251b0c1231ed1cf9fdfcf3924d06",
+  "source": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/remap-cache-r0p0032-v2/selected-remapped.jsonl",
+  "source_block_size": 512,
+  "source_blocks": 55641,
+  "source_rows_scanned": 3477,
+  "source_sha256": "f46c7d8ddbd146bafe00fbc173bf3f29039d8b96aa611f1ae4e2a0b9eac189f1",
+  "synthetic_fallback_contract": {
+    "different_16_block_hash_different_tokens": true,
+    "encoding": "injective base-(vocab_size-token_offset), 16 little-endian digits",
+    "runtime_identity_is_parent_sensitive": true,
+    "same_16_block_hash_same_tokens": true,
+    "token_offset": 1024,
+    "vocab_size": 151936
+  },
+  "target_16_blocks": 1766081,
+  "target_block_size": 16,
+  "total_decode_tokens": 494859,
+  "total_prefill_tokens": 28263677,
+  "upstream_remap_manifest": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/remap-cache-r0p0032-v2/manifest.json",
+  "upstream_remap_manifest_sha256": "8649b6acf10007e16a6b0eccebe95ee25a33481d6c3c0381ec8caf6dad05f12d",
+  "window": {
+    "duration_s": null,
+    "start_timestamp": null
+  },
+  "workload_mode": "prefill_decode"
+}

runs/frontier-code-trace-v0/results/paired-input-manifests/full-r0p0016-v1.json

@@ -0,0 +1,69 @@
+{
+  "block_contract": {
+    "input_length_mismatches": 0,
+    "runtime_identity_collisions": 0,
+    "runtime_to_source_relation_conflicts": 0,
+    "source_hash_count_mismatches": 0,
+    "source_runtime_granularity": "one 512-token source hash to 32 consecutive 16-token identities; final partial source block may have fewer",
+    "source_to_runtime_relation_conflicts": 0,
+    "unique_runtime_identities": 2613467,
+    "unique_runtime_relations": 0,
+    "unique_source_hash_relations": 0
+  },
+  "excluded_over_max_total_tokens": 0,
+  "frontier_csv": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/full-r0p0016-v1/frontier.csv",
+  "frontier_csv_sha256": "d785665c86a562c9d9a7492ad710e6393bb883bf8c3ad552a34ff18f22925043",
+  "mapping": "real prompt tokens: BLAKE2b-128(parent runtime identity, exact 16-token block); missing prompt fallback: source hashes first define deterministic 16-token content blocks, then the same parent-sensitive BLAKE2b-128 runtime identity contract is applied",
+  "max_total_tokens_filter": null,
+  "paired_row_vector_sha256": "d3d232ea41eadb507780b4830ce3e944ac88cd997d0a4f418f86ca71de1377bf",
+  "parent_validation": {
+    "enabled": false,
+    "links_checked": 0,
+    "links_outside_selected_stream": 0,
+    "mean_tail_nonreused_per_link": 0.0,
+    "parent_prefix_common_blocks": 0,
+    "parent_tail_nonreused_blocks": 0,
+    "rule": "contiguous common prefix required; mismatch tolerated only within parent trailing partial 512-token block (<= 33 runtime blocks)"
+  },
+  "prefix_cache_blocks": "complete 16-token blocks only; final partial block excluded",
+  "prompt_contract": {
+    "input_is_pre_remapped": true,
+    "real_prompt_requests": 0,
+    "synthetic_fallback_requests": 1718,
+    "tokenizer": null
+  },
+  "prompt_window": null,
+  "prompt_window_sha256": null,
+  "real_requests": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/full-r0p0016-v1/real_requests.jsonl",
+  "real_requests_sha256": "3261a15be3e1f7d5ecf8a86c03a9e0ff9ec69a3502e4fbc8e8b82b18b0854e1f",
+  "requests": 1718,
+  "rho": 0.0016,
+  "sampling_rule": "keep iff sampling_u <= rho (SimFid convention; rho=1 keeps all rows)",
+  "schema": "frontier-s3-real-remap-v2",
+  "selected_remapped": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/full-r0p0016-v1/selected-remapped.jsonl",
+  "selected_remapped_sha256": "4d5c66e5cad4efa4c500009630c6ca6069fdd71c9e1a7cd446c8869d84dc70f7",
+  "source": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/remap-cache-r0p0032-v2/selected-remapped.jsonl",
+  "source_block_size": 512,
+  "source_blocks": 104582,
+  "source_rows_scanned": 3477,
+  "source_sha256": "f46c7d8ddbd146bafe00fbc173bf3f29039d8b96aa611f1ae4e2a0b9eac189f1",
+  "synthetic_fallback_contract": {
+    "different_16_block_hash_different_tokens": true,
+    "encoding": "injective base-(vocab_size-token_offset), 16 little-endian digits",
+    "runtime_identity_is_parent_sensitive": true,
+    "same_16_block_hash_same_tokens": true,
+    "token_offset": 1024,
+    "vocab_size": 151936
+  },
+  "target_16_blocks": 3318510,
+  "target_block_size": 16,
+  "total_decode_tokens": 813493,
+  "total_prefill_tokens": 53108885,
+  "upstream_remap_manifest": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/remap-cache-r0p0032-v2/manifest.json",
+  "upstream_remap_manifest_sha256": "8649b6acf10007e16a6b0eccebe95ee25a33481d6c3c0381ec8caf6dad05f12d",
+  "window": {
+    "duration_s": null,
+    "start_timestamp": null
+  },
+  "workload_mode": "prefill_decode"
+}

runs/frontier-code-trace-v0/results/paired-input-manifests/prefill-only-r0p0032-v1.json

@@ -0,0 +1,69 @@
+{
+  "block_contract": {
+    "input_length_mismatches": 0,
+    "runtime_identity_collisions": 0,
+    "runtime_to_source_relation_conflicts": 0,
+    "source_hash_count_mismatches": 0,
+    "source_runtime_granularity": "one 512-token source hash to 32 consecutive 16-token identities; final partial source block may have fewer",
+    "source_to_runtime_relation_conflicts": 0,
+    "unique_runtime_identities": 2613467,
+    "unique_runtime_relations": 0,
+    "unique_source_hash_relations": 0
+  },
+  "excluded_over_max_total_tokens": 0,
+  "frontier_csv": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/prefill-only-r0p0032-v1/frontier.csv",
+  "frontier_csv_sha256": "8eeea3f44626e30e6da6ab58950c9a7f52e6cf6eb29c6bd2e35e1c25e4a9c5ce",
+  "mapping": "real prompt tokens: BLAKE2b-128(parent runtime identity, exact 16-token block); missing prompt fallback: source hashes first define deterministic 16-token content blocks, then the same parent-sensitive BLAKE2b-128 runtime identity contract is applied",
+  "max_total_tokens_filter": null,
+  "paired_row_vector_sha256": "40865068e02414612ba1cd4595894e20e85e01fd34d73f8660185552d531ecea",
+  "parent_validation": {
+    "enabled": false,
+    "links_checked": 0,
+    "links_outside_selected_stream": 0,
+    "mean_tail_nonreused_per_link": 0.0,
+    "parent_prefix_common_blocks": 0,
+    "parent_tail_nonreused_blocks": 0,
+    "rule": "contiguous common prefix required; mismatch tolerated only within parent trailing partial 512-token block (<= 33 runtime blocks)"
+  },
+  "prefix_cache_blocks": "complete 16-token blocks only; final partial block excluded",
+  "prompt_contract": {
+    "input_is_pre_remapped": true,
+    "real_prompt_requests": 0,
+    "synthetic_fallback_requests": 3477,
+    "tokenizer": null
+  },
+  "prompt_window": null,
+  "prompt_window_sha256": null,
+  "real_requests": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/prefill-only-r0p0032-v1/real_requests.jsonl",
+  "real_requests_sha256": "48a9a6f5f73fe4daebff5a2fba9aec732a89219d19a00d6811cf14e0cd5afa2c",
+  "requests": 3477,
+  "rho": 0.0032,
+  "sampling_rule": "keep iff sampling_u <= rho (SimFid convention; rho=1 keeps all rows)",
+  "schema": "frontier-s3-real-remap-v2",
+  "selected_remapped": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/prefill-only-r0p0032-v1/selected-remapped.jsonl",
+  "selected_remapped_sha256": "a9c31bb600ac6b637ded1c1aa4c84df5613bc3fbdc986bfe935f4ebe9aee0aff",
+  "source": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/remap-cache-r0p0032-v2/selected-remapped.jsonl",
+  "source_block_size": 512,
+  "source_blocks": 227960,
+  "source_rows_scanned": 3477,
+  "source_sha256": "f46c7d8ddbd146bafe00fbc173bf3f29039d8b96aa611f1ae4e2a0b9eac189f1",
+  "synthetic_fallback_contract": {
+    "different_16_block_hash_different_tokens": true,
+    "encoding": "injective base-(vocab_size-token_offset), 16 little-endian digits",
+    "runtime_identity_is_parent_sensitive": true,
+    "same_16_block_hash_same_tokens": true,
+    "token_offset": 1024,
+    "vocab_size": 151936
+  },
+  "target_16_blocks": 7237642,
+  "target_block_size": 16,
+  "total_decode_tokens": 3477,
+  "total_prefill_tokens": 115828371,
+  "upstream_remap_manifest": "/home/admin/cpfs/wjh/aituner/aituner-code-trace-fbaa909/runs/frontier-code-trace-v0/inputs/remap-cache-r0p0032-v2/manifest.json",
+  "upstream_remap_manifest_sha256": "8649b6acf10007e16a6b0eccebe95ee25a33481d6c3c0381ec8caf6dad05f12d",
+  "window": {
+    "duration_s": null,
+    "start_timestamp": null
+  },
+  "workload_mode": "prefill_only"
+}

runs/frontier-code-trace-v0/results/prefill-only-calibration-summary.json

@@ -0,0 +1,539 @@
+{
+  "attention_profile_sha256": "fbcf7e1f95789a6f6d771e24d1fc60958b7daf04eb0db260d27869a19d71d550",
+  "cell_version": "v1",
+  "cells": [
+    {
+      "decode_batch": {
+        "histogram": {},
+        "max": 0,
+        "share_gt_1": 0.0,
+        "share_gt_4": 0.0,
+        "stages": 0
+      },
+      "drain_fraction": 0.0009942595174267852,
+      "drain_tail_s": 3.638989833782034,
+      "offered_load": {
+        "decode_tokens_per_s": 0.0633879781420765,
+        "prefill_tokens_per_s_after_prefix": 884.2945355191257,
+        "prefill_tokens_per_s_raw": 2459.7437158469947,
+        "requests_per_s": 0.0633879781420765
+      },
+      "prefix_cache_hit_ratio": 0.6406200282639031,
+      "requests": 232,
+      "rho": 0.0002,
+      "subcritical_gate": true,
+      "topology": "tp2_mns16",
+      "tpot_ms": {
+        "count": 0,
+        "max": null,
+        "mean": null,
+        "min": null,
+        "p50": null,
+        "p90": null,
+        "p95": null,
+        "p99": null
+      },
+      "trace_sha256": "573cf4b21a3c39789b87eb92a34c8d8c95a8d0d79f769b05719884590f48c639",
+      "ttft_ms": {
+        "count": 232,
+        "max": 22394.384315265255,
+        "mean": 2016.4229573072757,
+        "min": 90.95485497891787,
+        "p50": 876.6052006358223,
+        "p90": 5597.657549404108,
+        "p95": 9955.05948414866,
+        "p99": 15612.433918519466
+      },
+      "waiting_ms": {
+        "count": 232,
+        "max": 13155.607757982125,
+        "mean": 336.5077998372315,
+        "min": 0.0,
+        "p50": 0.0,
+        "p90": 0.0,
+        "p95": 1592.2631189191934,
+        "p99": 7965.408809250165
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {},
+        "max": 0,
+        "share_gt_1": 0.0,
+        "share_gt_4": 0.0,
+        "stages": 0
+      },
+      "drain_fraction": 0.0005535579625697598,
+      "drain_tail_s": 2.0260221430053207,
+      "offered_load": {
+        "decode_tokens_per_s": 0.0633879781420765,
+        "prefill_tokens_per_s_after_prefix": 880.7972677595628,
+        "prefill_tokens_per_s_raw": 2459.7437158469947,
+        "requests_per_s": 0.0633879781420765
+      },
+      "prefix_cache_hit_ratio": 0.6420421114379927,
+      "requests": 232,
+      "rho": 0.0002,
+      "subcritical_gate": true,
+      "topology": "tp4_mns16",
+      "tpot_ms": {
+        "count": 0,
+        "max": null,
+        "mean": null,
+        "min": null,
+        "p50": null,
+        "p90": null,
+        "p95": null,
+        "p99": null
+      },
+      "trace_sha256": "573cf4b21a3c39789b87eb92a34c8d8c95a8d0d79f769b05719884590f48c639",
+      "ttft_ms": {
+        "count": 232,
+        "max": 12080.695954866542,
+        "mean": 1097.5137410350026,
+        "min": 72.1560867923472,
+        "p50": 516.1309269643652,
+        "p90": 2564.188634116818,
+        "p95": 5585.788369509169,
+        "p99": 8804.50146872767
+      },
+      "waiting_ms": {
+        "count": 232,
+        "max": 7151.845512636555,
+        "mean": 99.8249313518456,
+        "min": 0.0,
+        "p50": 0.0,
+        "p90": 0.0,
+        "p95": 295.97096758665066,
+        "p99": 2261.809117635907
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {},
+        "max": 0,
+        "share_gt_1": 0.0,
+        "share_gt_4": 0.0,
+        "stages": 0
+      },
+      "drain_fraction": 0.0013382625998707982,
+      "drain_tail_s": 4.898041115527121,
+      "offered_load": {
+        "decode_tokens_per_s": 0.13633879781420766,
+        "prefill_tokens_per_s_after_prefix": 1794.7636612021859,
+        "prefill_tokens_per_s_raw": 5620.918852459016,
+        "requests_per_s": 0.13633879781420766
+      },
+      "prefix_cache_hit_ratio": 0.6808258903220888,
+      "requests": 499,
+      "rho": 0.0004,
+      "subcritical_gate": true,
+      "topology": "tp2_mns16",
+      "tpot_ms": {
+        "count": 0,
+        "max": null,
+        "mean": null,
+        "min": null,
+        "p50": null,
+        "p90": null,
+        "p95": null,
+        "p99": null
+      },
+      "trace_sha256": "b27aec81dd9baa832176679b8e84333720fd70946882fbb079de7b912332feda",
+      "ttft_ms": {
+        "count": 499,
+        "max": 28160.869292443294,
+        "mean": 2470.467680714121,
+        "min": 90.95485497891787,
+        "p50": 877.1610873113787,
+        "p90": 7449.431996229117,
+        "p95": 12179.038589710988,
+        "p99": 18584.71742409069
+      },
+      "waiting_ms": {
+        "count": 499,
+        "max": 25191.932798227754,
+        "mean": 864.2429863694354,
+        "min": 0.0,
+        "p50": 0.0,
+        "p90": 2060.9470903828583,
+        "p95": 5987.232655719128,
+        "p99": 13173.044246174226
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {},
+        "max": 0,
+        "share_gt_1": 0.0,
+        "share_gt_4": 0.0,
+        "stages": 0
+      },
+      "drain_fraction": 0.0005723542337033552,
+      "drain_tail_s": 2.09481649535428,
+      "offered_load": {
+        "decode_tokens_per_s": 0.13633879781420766,
+        "prefill_tokens_per_s_after_prefix": 1734.9516393442623,
+        "prefill_tokens_per_s_raw": 5620.918852459016,
+        "requests_per_s": 0.13633879781420766
+      },
+      "prefix_cache_hit_ratio": 0.691468838921361,
+      "requests": 499,
+      "rho": 0.0004,
+      "subcritical_gate": true,
+      "topology": "tp4_mns16",
+      "tpot_ms": {
+        "count": 0,
+        "max": null,
+        "mean": null,
+        "min": null,
+        "p50": null,
+        "p90": null,
+        "p95": null,
+        "p99": null
+      },
+      "trace_sha256": "b27aec81dd9baa832176679b8e84333720fd70946882fbb079de7b912332feda",
+      "ttft_ms": {
+        "count": 499,
+        "max": 14638.743702257216,
+        "mean": 1190.143380934895,
+        "min": 72.1560867923472,
+        "p50": 510.84479136716254,
+        "p90": 3181.3298830905296,
+        "p95": 5265.102348068058,
+        "p99": 10414.488385287405
+      },
+      "waiting_ms": {
+        "count": 499,
+        "max": 12819.0035203138,
+        "mean": 255.23631772331117,
+        "min": 0.0,
+        "p50": 0.0,
+        "p90": 96.2998426639842,
+        "p95": 1728.7002592128508,
+        "p99": 5892.272316403113
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {},
+        "max": 0,
+        "share_gt_1": 0.0,
+        "share_gt_4": 0.0,
+        "stages": 0
+      },
+      "drain_fraction": 0.00023484544575858607,
+      "drain_tail_s": 0.859534331476425,
+      "offered_load": {
+        "decode_tokens_per_s": 0.23743169398907105,
+        "prefill_tokens_per_s_after_prefix": 3120.9784153005467,
+        "prefill_tokens_per_s_raw": 7722.316120218579,
+        "requests_per_s": 0.23743169398907105
+      },
+      "prefix_cache_hit_ratio": 0.5959839894093193,
+      "requests": 869,
+      "rho": 0.0008,
+      "subcritical_gate": true,
+      "topology": "tp2_mns16",
+      "tpot_ms": {
+        "count": 0,
+        "max": null,
+        "mean": null,
+        "min": null,
+        "p50": null,
+        "p90": null,
+        "p95": null,
+        "p99": null
+      },
+      "trace_sha256": "a440006d71f928053da5a88af7e00003169a1f95e05ee8328c5c3ca697ae0fe9",
+      "ttft_ms": {
+        "count": 869,
+        "max": 31384.57081632896,
+        "mean": 3262.922770577521,
+        "min": 80.48812343014333,
+        "p50": 869.0654534193527,
+        "p90": 10573.947661327758,
+        "p95": 15296.650926758037,
+        "p99": 21984.870012304014
+      },
+      "waiting_ms": {
+        "count": 869,
+        "max": 30294.53558294381,
+        "mean": 1770.386177703944,
+        "min": 0.0,
+        "p50": 0.0,
+        "p90": 6890.822699274853,
+        "p95": 11429.513143558317,
+        "p99": 18054.967406463602
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {},
+        "max": 0,
+        "share_gt_1": 0.0,
+        "share_gt_4": 0.0,
+        "stages": 0
+      },
+      "drain_fraction": 0.00014906636399725652,
+      "drain_tail_s": 0.5455828922299588,
+      "offered_load": {
+        "decode_tokens_per_s": 0.23743169398907105,
+        "prefill_tokens_per_s_after_prefix": 2943.2210382513663,
+        "prefill_tokens_per_s_raw": 7722.316120218579,
+        "requests_per_s": 0.23743169398907105
+      },
+      "prefix_cache_hit_ratio": 0.6190078484508921,
+      "requests": 869,
+      "rho": 0.0008,
+      "subcritical_gate": true,
+      "topology": "tp4_mns16",
+      "tpot_ms": {
+        "count": 0,
+        "max": null,
+        "mean": null,
+        "min": null,
+        "p50": null,
+        "p90": null,
+        "p95": null,
+        "p99": null
+      },
+      "trace_sha256": "a440006d71f928053da5a88af7e00003169a1f95e05ee8328c5c3ca697ae0fe9",
+      "ttft_ms": {
+        "count": 869,
+        "max": 16057.04136311897,
+        "mean": 1275.40149118524,
+        "min": 67.46058986800563,
+        "p50": 428.09781420987747,
+        "p90": 3685.320778708873,
+        "p95": 6286.041945787507,
+        "p99": 10430.684729989898
+      },
+      "waiting_ms": {
+        "count": 869,
+        "max": 15053.141906136261,
+        "mean": 421.3322223628933,
+        "min": 0.0,
+        "p50": 0.0,
+        "p90": 585.9887748068106,
+        "p95": 3090.558970424443,
+        "p99": 8471.886860828818
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {},
+        "max": 0,
+        "share_gt_1": 0.0,
+        "share_gt_4": 0.0,
+        "stages": 0
+      },
+      "drain_fraction": 0.012967957147314541,
+      "drain_tail_s": 47.46272315917122,
+      "offered_load": {
+        "decode_tokens_per_s": 0.46939890710382515,
+        "prefill_tokens_per_s_after_prefix": 6538.567486338798,
+        "prefill_tokens_per_s_raw": 14510.624316939891,
+        "requests_per_s": 0.46939890710382515
+      },
+      "prefix_cache_hit_ratio": 0.5495261427568396,
+      "requests": 1718,
+      "rho": 0.0016,
+      "subcritical_gate": true,
+      "topology": "tp2_mns16",
+      "tpot_ms": {
+        "count": 0,
+        "max": null,
+        "mean": null,
+        "min": null,
+        "p50": null,
+        "p90": null,
+        "p95": null,
+        "p99": null
+      },
+      "trace_sha256": "60080a3b70c01806782a0abd72c873536aae54fbd9c5ef70d1d5a2ae7a1338ea",
+      "ttft_ms": {
+        "count": 1718,
+        "max": 66578.48546737114,
+        "mean": 8771.952477826575,
+        "min": 80.48812343014333,
+        "p50": 3392.0416080366067,
+        "p90": 24475.218749701922,
+        "p95": 33824.30978066004,
+        "p99": 48636.304823714425
+      },
+      "waiting_ms": {
+        "count": 1718,
+        "max": 65625.6292665521,
+        "mean": 7210.059307774163,
+        "min": 0.0,
+        "p50": 1210.4389057114986,
+        "p90": 21454.74688922785,
+        "p95": 31447.91008182273,
+        "p99": 47140.04440652531
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {},
+        "max": 0,
+        "share_gt_1": 0.0,
+        "share_gt_4": 0.0,
+        "stages": 0
+      },
+      "drain_fraction": 0.0024829272772107624,
+      "drain_tail_s": 9.087513834591391,
+      "offered_load": {
+        "decode_tokens_per_s": 0.46939890710382515,
+        "prefill_tokens_per_s_after_prefix": 6046.31393442623,
+        "prefill_tokens_per_s_raw": 14510.624316939891,
+        "requests_per_s": 0.46939890710382515
+      },
+      "prefix_cache_hit_ratio": 0.5834579374478305,
+      "requests": 1718,
+      "rho": 0.0016,
+      "subcritical_gate": true,
+      "topology": "tp4_mns16",
+      "tpot_ms": {
+        "count": 0,
+        "max": null,
+        "mean": null,
+        "min": null,
+        "p50": null,
+        "p90": null,
+        "p95": null,
+        "p99": null
+      },
+      "trace_sha256": "60080a3b70c01806782a0abd72c873536aae54fbd9c5ef70d1d5a2ae7a1338ea",
+      "ttft_ms": {
+        "count": 1718,
+        "max": 22740.47242973984,
+        "mean": 1982.5181747731513,
+        "min": 67.46058986800563,
+        "p50": 563.4391554845024,
+        "p90": 6378.403293449832,
+        "p95": 8710.966056077563,
+        "p99": 15357.245352912887
+      },
+      "waiting_ms": {
+        "count": 1718,
+        "max": 18888.109661448652,
+        "mean": 1110.9868411863508,
+        "min": 0.0,
+        "p50": 0.0,
+        "p90": 4253.667468357389,
+        "p95": 6886.657874642189,
+        "p99": 13397.793338046433
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {},
+        "max": 0,
+        "share_gt_1": 0.0,
+        "share_gt_4": 0.0,
+        "stages": 0
+      },
+      "drain_fraction": 0.37830325420435085,
+      "drain_tail_s": 1384.5899103879242,
+      "offered_load": {
+        "decode_tokens_per_s": 0.95,
+        "prefill_tokens_per_s_after_prefix": 14302.829234972678,
+        "prefill_tokens_per_s_raw": 31647.095901639343,
+        "requests_per_s": 0.95
+      },
+      "prefix_cache_hit_ratio": 0.548175911436349,
+      "requests": 3477,
+      "rho": 0.0032,
+      "subcritical_gate": false,
+      "topology": "tp2_mns16",
+      "tpot_ms": {
+        "count": 0,
+        "max": null,
+        "mean": null,
+        "min": null,
+        "p50": null,
+        "p90": null,
+        "p95": null,
+        "p99": null
+      },
+      "trace_sha256": "8eeea3f44626e30e6da6ab58950c9a7f52e6cf6eb29c6bd2e35e1c25e4a9c5ce",
+      "ttft_ms": {
+        "count": 3477,
+        "max": 1396505.4377501393,
+        "mean": 713718.2923315652,
+        "min": 523.4377250571841,
+        "p50": 752636.5425455952,
+        "p90": 1219928.0000770227,
+        "p95": 1265657.4963084226,
+        "p99": 1364995.269473642
+      },
+      "waiting_ms": {
+        "count": 3477,
+        "max": 1395552.5815493201,
+        "mean": 711986.6468225399,
+        "min": 0.0,
+        "p50": 751006.927897161,
+        "p90": 1218640.0576069213,
+        "p95": 1264383.90901991,
+        "p99": 1357836.932620136
+      }
+    },
+    {
+      "decode_batch": {
+        "histogram": {},
+        "max": 0,
+        "share_gt_1": 0.0,
+        "share_gt_4": 0.0,
+        "stages": 0
+      },
+      "drain_fraction": 0.0024829272772107624,
+      "drain_tail_s": 9.087513834591391,
+      "offered_load": {
+        "decode_tokens_per_s": 0.95,
+        "prefill_tokens_per_s_after_prefix": 12622.671857923497,
+        "prefill_tokens_per_s_raw": 31647.095901639343,
+        "requests_per_s": 0.95
+      },
+      "prefix_cache_hit_ratio": 0.6012782892549811,
+      "requests": 3477,
+      "rho": 0.0032,
+      "subcritical_gate": true,
+      "topology": "tp4_mns16",
+      "tpot_ms": {
+        "count": 0,
+        "max": null,
+        "mean": null,
+        "min": null,
+        "p50": null,
+        "p90": null,
+        "p95": null,
+        "p99": null
+      },
+      "trace_sha256": "8eeea3f44626e30e6da6ab58950c9a7f52e6cf6eb29c6bd2e35e1c25e4a9c5ce",
+      "ttft_ms": {
+        "count": 3477,
+        "max": 43733.59998241585,
+        "mean": 6397.374766665999,
+        "min": 67.46058986814774,
+        "p50": 3276.4329854612697,
+        "p90": 17833.72132484574,
+        "p95": 23705.880006424282,
+        "p99": 36214.16656225671
+      },
+      "waiting_ms": {
+        "count": 3477,
+        "max": 40748.976762736216,
+        "mean": 5494.564635817035,
+        "min": 0.0,
+        "p50": 1963.6734690996036,
+        "p90": 16680.754893288453,
+        "p95": 22203.49896551297,
+        "p99": 34692.440763376595
+      }
+    }
+  ],
+  "schema": "frontier-code-trace-calibration-summary-v1",
+  "subcritical_rule": "drain tail <= 10% of the 3660s arrival window",
+  "workload_mode": "prefill_only"
+}

runs/frontier-code-trace-v0/run_flashattn_code_longctx.sh

@@ -0,0 +1,88 @@
+#!/usr/bin/env bash
+# Long-context FlashAttention profile grid for the code-trace campaign.
+
+set -euo pipefail
+
+TP="${TP:?TP must be set to 1, 2, or 4}"
+case "${TP}" in
+  1|2|4) ;;
+  *) echo "ERROR: invalid TP=${TP}" >&2; exit 1 ;;
+esac
+
+OUTPUT_ROOT="${OUTPUT_ROOT:?OUTPUT_ROOT must be set}"
+MAX_MODEL_LEN="${MAX_MODEL_LEN:-147456}"
+ALLOW_LONG_CONTEXT_PROFILE="${ALLOW_LONG_CONTEXT_PROFILE:-false}"
+VENV_ROOT="${VENV_ROOT:-/home/admin/cpfs/wjh/venvs/vllm-0.20.0-cu129-workload-regime-v2}"
+VLLM_SOURCE="${VLLM_SOURCE:-/home/admin/cpfs/wjh/agentic-kv/third_party/vllm_v20_build}"
+MODEL="${MODEL:-/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B}"
+CAMPAIGN_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROFILE_SCRIPT="${CAMPAIGN_ROOT}/../frontier-qwen30-vllm020-profile-v1/profile_vllm020_flashattn.py"
+
+# Development window:
+#   ISL p90=88,224, p99=125,803; sampled rho<=0.0032 max total=137,016.
+# Tail query distribution modulo 8192:
+#   p10=495, p25=1477, p50=3594, p75=5917, p90=7246.
+# The full-chunk grid reaches 131,072 tokens of existing context; tail shapes
+# cover query-size variation at representative long contexts.
+BATCH_SPECS=(
+  q8ks48k q8ks64k q8ks80k q8ks96k q8ks112k q8ks128k q8ks136k
+  q512s128k q2ks66k q4ks100k q6ks134k
+  q1ks8k q512s4k
+)
+
+mkdir -p "${OUTPUT_ROOT}/logs" "${OUTPUT_ROOT}/provenance" "${OUTPUT_ROOT}/raw"
+exec > >(tee -a "${OUTPUT_ROOT}/logs/code-longctx-grid.log") 2>&1
+
+IFS=',' read -r -a GPU_IDS <<< "${CUDA_VISIBLE_DEVICES:?one allocated fleet GPU is required}"
+if [[ "${#GPU_IDS[@]}" -ne 1 ]]; then
+  echo "ERROR: expected exactly one GPU, got ${CUDA_VISIBLE_DEVICES}" >&2
+  exit 1
+fi
+
+test -x "${VENV_ROOT}/bin/python"
+test -f "${VLLM_SOURCE}/benchmarks/attention_benchmarks/runner.py"
+test -f "${MODEL}/config.json"
+test -f "${PROFILE_SCRIPT}"
+if [[ "${ALLOW_LONG_CONTEXT_PROFILE}" != "true" ]]; then
+  echo "ERROR: set ALLOW_LONG_CONTEXT_PROFILE=true after reviewing the model's 40960-token RoPE contract" >&2
+  exit 1
+fi
+export VLLM_ALLOW_LONG_MAX_MODEL_LEN=1
+
+echo "PROFILE_LAUNCH_ECHO host=$(hostname) gpu=${CUDA_VISIBLE_DEVICES} tp=${TP} max_model_len=${MAX_MODEL_LEN} specs=${BATCH_SPECS[*]}"
+date -u +"START_UTC=%Y-%m-%dT%H:%M:%SZ"
+nvidia-smi --query-gpu=index,name,driver_version,memory.used,utilization.gpu \
+  --format=csv,noheader
+git -C "${CAMPAIGN_ROOT}/../.." rev-parse HEAD \
+  > "${OUTPUT_ROOT}/provenance/aituner.commit"
+git -C "${VLLM_SOURCE}" rev-parse HEAD \
+  > "${OUTPUT_ROOT}/provenance/vllm-source.commit"
+sha256sum "${PROFILE_SCRIPT}" "${BASH_SOURCE[0]}" \
+  > "${OUTPUT_ROOT}/provenance/source.sha256"
+uv pip freeze --python "${VENV_ROOT}/bin/python" \
+  > "${OUTPUT_ROOT}/provenance/pip-freeze.txt"
+printf '%s\n' "${BATCH_SPECS[@]}" \
+  > "${OUTPUT_ROOT}/provenance/batch-specs.txt"
+printf '%s\n' "${MAX_MODEL_LEN}" \
+  > "${OUTPUT_ROOT}/provenance/max-model-len.txt"
+printf '%s\n' "${VLLM_ALLOW_LONG_MAX_MODEL_LEN}" \
+  > "${OUTPUT_ROOT}/provenance/vllm-allow-long-max-model-len.txt"
+
+timeout --signal=TERM --kill-after=30s 1800 \
+  "${VENV_ROOT}/bin/python" "${PROFILE_SCRIPT}" \
+  --vllm-source "${VLLM_SOURCE}" \
+  --model "${MODEL}" \
+  --output "${OUTPUT_ROOT}/raw/flashattn-code-longctx-tp${TP}.json" \
+  --tp "${TP}" \
+  --batch-specs "${BATCH_SPECS[@]}" \
+  --warmup-iters 5 \
+  --repeats 10 \
+  --max-model-len "${MAX_MODEL_LEN}" \
+  --profile-kv-update
+
+test -s "${OUTPUT_ROOT}/raw/flashattn-code-longctx-tp${TP}.json"
+sha256sum "${OUTPUT_ROOT}/raw/flashattn-code-longctx-tp${TP}.json" \
+  "${OUTPUT_ROOT}/provenance"/* > "${OUTPUT_ROOT}/artifacts.sha256"
+nvidia-smi --query-gpu=index,name,memory.used,utilization.gpu --format=csv,noheader
+date -u +"END_UTC=%Y-%m-%dT%H:%M:%SZ"
+echo "FLASHATTN_CODE_LONGCTX_COMPLETE tp=${TP} cases=${#BATCH_SPECS[@]}"

runs/frontier-code-trace-v0/run_max_model_len_smoke.sh

@@ -0,0 +1,143 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+PORT="${1:-8123}"
+CONTROL_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(cd "${CONTROL_ROOT}/../.." && pwd)"
+VENV_ROOT="${VENV_ROOT:-/home/admin/cpfs/wjh/venvs/vllm-0.20.0-cu129-workload-regime-v2}"
+MODEL_ROOT="${MODEL_ROOT:-/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B}"
+OUTPUT_ROOT="${OUTPUT_ROOT:-${CONTROL_ROOT}/outputs/max-model-len-smoke}"
+SERVED_MODEL="qwen3-30b-code-maxlen-smoke"
+MAX_MODEL_LEN=147456
+SERVER_PID=""
+
+[[ "${ALLOW_LONG_CONTEXT_SERVER:-false}" == "true" ]] || {
+  echo "ERROR: set ALLOW_LONG_CONTEXT_SERVER=true to acknowledge the model's native 40960 limit" >&2
+  exit 1
+}
+[[ "${CUDA_VISIBLE_DEVICES:-}" == "0,1,2,3" ]] || {
+  echo "ERROR: this TP4 smoke requires CUDA_VISIBLE_DEVICES=0,1,2,3" >&2
+  exit 1
+}
+for path in \
+  "${VENV_ROOT}/bin/python" \
+  "${VENV_ROOT}/bin/vllm" \
+  "${MODEL_ROOT}/config.json" \
+  "${REPO_ROOT}/runs/frontier-s3-real-v0/qwen30_prefill_client.py"; do
+  [[ -e "${path}" ]] || { echo "ERROR: missing ${path}" >&2; exit 1; }
+done
+[[ ! -e "${OUTPUT_ROOT}" ]] || {
+  echo "ERROR: refusing to overwrite ${OUTPUT_ROOT}" >&2
+  exit 1
+}
+mkdir -p "${OUTPUT_ROOT}/logs" "${OUTPUT_ROOT}/results" "${OUTPUT_ROOT}/provenance"
+
+cleanup() {
+  if [[ -n "${SERVER_PID}" ]] && kill -0 "${SERVER_PID}" 2>/dev/null; then
+    kill -TERM -- "-${SERVER_PID}" 2>/dev/null || true
+    for _ in $(seq 1 30); do
+      kill -0 "${SERVER_PID}" 2>/dev/null || break
+      sleep 1
+    done
+    kill -KILL -- "-${SERVER_PID}" 2>/dev/null || true
+  fi
+}
+trap cleanup EXIT INT TERM
+
+nvidia-smi \
+  --query-gpu=index,name,memory.used,utilization.gpu,ecc.errors.uncorrected.aggregate.total \
+  --format=csv,noheader,nounits > "${OUTPUT_ROOT}/provenance/gpus.before.csv"
+[[ "$(wc -l < "${OUTPUT_ROOT}/provenance/gpus.before.csv")" -eq 8 ]] || {
+  echo "ERROR: expected eight GPUs on host" >&2
+  exit 1
+}
+awk -F, '
+  $2 !~ /NVIDIA H20/ || $3 + 0 != 0 || $4 + 0 != 0 || $5 + 0 != 0 { bad = 1 }
+  END { exit bad }
+' "${OUTPUT_ROOT}/provenance/gpus.before.csv" || {
+  echo "ERROR: all eight H20 GPUs must be idle and healthy before the smoke" >&2
+  exit 1
+}
+if nvidia-smi --query-compute-apps=pid --format=csv,noheader,nounits \
+  | grep -Eq '^[[:space:]]*[0-9]+'; then
+  echo "ERROR: compute process appeared after the idle probe" >&2
+  exit 1
+fi
+
+export TOKENIZERS_PARALLELISM=false
+export VLLM_USE_V1=1
+export VLLM_ALLOW_LONG_MAX_MODEL_LEN=1
+export HF_HUB_OFFLINE=1
+export TRANSFORMERS_OFFLINE=1
+export FLASHINFER_WORKSPACE_BASE="/tmp/wjh/frontier-code-maxlen-smoke"
+mkdir -p "${FLASHINFER_WORKSPACE_BASE}"
+ulimit -n 65536
+
+setsid "${VENV_ROOT}/bin/vllm" serve "${MODEL_ROOT}" \
+  --host 127.0.0.1 \
+  --port "${PORT}" \
+  --served-model-name "${SERVED_MODEL}" \
+  --tensor-parallel-size 4 \
+  --gpu-memory-utilization 0.92 \
+  --hf-overrides "{\"max_position_embeddings\":${MAX_MODEL_LEN}}" \
+  --max-model-len "${MAX_MODEL_LEN}" \
+  --max-num-batched-tokens 8192 \
+  --max-num-seqs 16 \
+  --enable-prefix-caching \
+  --block-size 16 \
+  --enable-chunked-prefill \
+  --no-enable-log-requests \
+  > "${OUTPUT_ROOT}/logs/server.log" 2>&1 &
+SERVER_PID=$!
+
+READY=0
+for _ in $(seq 1 900); do
+  if curl -fsS --max-time 2 "http://127.0.0.1:${PORT}/v1/models" \
+    > "${OUTPUT_ROOT}/results/models.json" 2>/dev/null; then
+    READY=1
+    break
+  fi
+  kill -0 "${SERVER_PID}" 2>/dev/null || {
+    tail -200 "${OUTPUT_ROOT}/logs/server.log"
+    exit 1
+  }
+  sleep 3
+done
+[[ "${READY}" -eq 1 ]] || {
+  echo "ERROR: server readiness timeout" >&2
+  exit 1
+}
+
+run_shape() {
+  local label="$1"
+  local input_tokens="$2"
+  local output_tokens="$3"
+  "${VENV_ROOT}/bin/python" \
+    "${REPO_ROOT}/runs/frontier-s3-real-v0/qwen30_prefill_client.py" \
+    --port "${PORT}" \
+    --served-model "${SERVED_MODEL}" \
+    --model-path "${MODEL_ROOT}" \
+    --rate 1 \
+    --requests 1 \
+    --input-tokens "${input_tokens}" \
+    --output-tokens "${output_tokens}" \
+    --timeout-seconds 1800 \
+    --output "${OUTPUT_ROOT}/results/${label}.json"
+}
+
+run_shape warmup 512 1
+run_shape code-p50 20051 78
+run_shape code-candidate-p99 119702 68
+run_shape code-candidate-max 136774 242
+
+cleanup
+SERVER_PID=""
+nvidia-smi --query-gpu=index,name,memory.used,utilization.gpu \
+  --format=csv,noheader,nounits > "${OUTPUT_ROOT}/provenance/gpus.after.csv"
+sha256sum \
+  "${BASH_SOURCE[0]}" \
+  "${REPO_ROOT}/runs/frontier-s3-real-v0/qwen30_prefill_client.py" \
+  "${MODEL_ROOT}/config.json" \
+  > "${OUTPUT_ROOT}/provenance/sources.sha256"
+echo "MAX_MODEL_LEN_SMOKE_COMPLETE output=${OUTPUT_ROOT}"

runs/frontier-code-trace-v0/summarize_calibration.py

@@ -0,0 +1,102 @@
+#!/usr/bin/env python3
+"""Compact code-trace Frontier rho calibration into a reviewable artifact."""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import re
+from pathlib import Path
+
+
+CELL_RE = re.compile(
+    r"r(?P<rho>[0-9p]+)-tp(?P<tp>[24])-(?P<version>v[0-9]+)"
+)
+ATTENTION_FLAG = "--random_forrest_execution_time_predictor_config_atten_input_file"
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--sim-root", type=Path, required=True)
+    parser.add_argument("--profile-manifest", type=Path, required=True)
+    parser.add_argument(
+        "--workload-mode",
+        choices=("prefill_decode", "prefill_only"),
+        default="prefill_decode",
+    )
+    parser.add_argument("--cell-version", default="v3")
+    parser.add_argument("--output", type=Path, required=True)
+    return parser.parse_args()
+
+
+def sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as stream:
+        for chunk in iter(lambda: stream.read(1024 * 1024), b""):
+            digest.update(chunk)
+    return digest.hexdigest()
+
+
+def main() -> None:
+    args = parse_args()
+    profile = json.loads(args.profile_manifest.read_text())
+    expected_profile_sha = profile["output_sha256"]
+    cells = []
+    for root in sorted(args.sim_root.iterdir()):
+        match = CELL_RE.fullmatch(root.name)
+        if (
+            match is None
+            or match.group("version") != args.cell_version
+            or not (root / "summary.json").is_file()
+        ):
+            continue
+        manifest = json.loads((root / "manifest.json").read_text())
+        observed_profile_sha = manifest.get("attention_profile_sha256")
+        if observed_profile_sha is None:
+            argv = manifest["argv"]
+            profile_path = Path(argv[argv.index(ATTENTION_FLAG) + 1])
+            observed_profile_sha = sha256(profile_path)
+        if observed_profile_sha != expected_profile_sha:
+            raise ValueError(
+                f"{root}: attention profile {observed_profile_sha} "
+                f"!= profile-v6 {expected_profile_sha}"
+            )
+        summary = json.loads((root / "summary.json").read_text())
+        drain_fraction = (
+            summary["drain"]["tail_after_last_arrival_s"] / summary["duration_s"]
+        )
+        cells.append(
+            {
+                "topology": f"tp{match.group('tp')}_mns16",
+                "rho": float(match.group("rho").replace("p", ".")),
+                "requests": summary["requests"],
+                "offered_load": summary["offered_load"],
+                "waiting_ms": summary["latency_ms"]["waiting"],
+                "ttft_ms": summary["latency_ms"]["ttft"],
+                "tpot_ms": summary["latency_ms"]["tpot"],
+                "decode_batch": summary["decode_batch"],
+                "drain_tail_s": summary["drain"]["tail_after_last_arrival_s"],
+                "drain_fraction": drain_fraction,
+                "subcritical_gate": drain_fraction <= 0.1,
+                "prefix_cache_hit_ratio": summary["prefix_cache"]["hit_ratio"],
+                "trace_sha256": manifest["trace_sha256"],
+            }
+        )
+    if not cells:
+        raise ValueError(f"no completed v3 calibration cells below {args.sim_root}")
+    payload = {
+        "schema": "frontier-code-trace-calibration-summary-v1",
+        "workload_mode": args.workload_mode,
+        "cell_version": args.cell_version,
+        "attention_profile_sha256": expected_profile_sha,
+        "subcritical_rule": "drain tail <= 10% of the 3660s arrival window",
+        "cells": cells,
+    }
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
+    print(json.dumps({"cells": len(cells), "output": str(args.output)}, sort_keys=True))
+
+
+if __name__ == "__main__":
+    main()

runs/frontier-code-trace-v0/test_code_trace_preflight.py

@@ -20,6 +20,21 @@ class CodeTracePreflightTest(unittest.TestCase):
             source = root / "051315-051317.jsonl"
             with source.open("w") as stream:
                 for index in range(4501):
+                    if index == 100:
+                        stream.write(
+                            json.dumps(
+                                {
+                                    "chat_id": index,
+                                    "parent_chat_id": -1,
+                                    "timestamp": float(index),
+                                    "input_length": 0,
+                                    "output_length": 0,
+                                    "hash_ids": [],
+                                }
+                            )
+                            + "\n"
+                        )
+                        continue
                     input_tokens = 513 if index % 2 else 512
                     stream.write(
                         json.dumps(
@@ -53,6 +68,7 @@ class CodeTracePreflightTest(unittest.TestCase):
             self.assertEqual(
                 payload["selected"]["hash_contract"]["exact_source_block_size"], 512
             )
+            self.assertEqual(payload["selected"]["invalid_zero_token_rows"], 1)
             self.assertEqual(payload["max_model_len_recommendation"], 40960)
             output = root / "window"
             subprocess.run(
@@ -74,6 +90,7 @@ class CodeTracePreflightTest(unittest.TestCase):
         self.assertEqual(manifest["requests"], len(rows))
         self.assertGreaterEqual(manifest["duration_s"], 3600)
         self.assertEqual(rows[0]["sampling_u"], rows[1]["sampling_u"])
+        self.assertNotIn(100, {row["source_index"] for row in rows})
 
 
 if __name__ == "__main__":

runs/frontier-code-trace-v0/test_frontier_code_replay.py

@@ -0,0 +1,114 @@
+#!/usr/bin/env python3
+
+from __future__ import annotations
+
+import csv
+import importlib.util
+import json
+import sys
+import tempfile
+import unittest
+from pathlib import Path
+
+
+ROOT = Path(__file__).resolve().parent
+S3_REAL = ROOT.parent / "frontier-s3-real-v0"
+
+
+def load_replay_module():
+    sys.path.insert(0, str(S3_REAL))
+    spec = importlib.util.spec_from_file_location(
+        "frontier_prefix_replay", S3_REAL / "run_frontier_prefix_replay.py"
+    )
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module
+
+
+class FrontierCodeReplayTest(unittest.TestCase):
+    def test_summary_excludes_undefined_single_token_tpot(self) -> None:
+        module = load_replay_module()
+        with tempfile.TemporaryDirectory() as tmp:
+            root = Path(tmp)
+            trace = root / "frontier.csv"
+            with trace.open("w", newline="") as stream:
+                writer = csv.DictWriter(
+                    stream,
+                    fieldnames=(
+                        "arrived_at",
+                        "num_prefill_tokens",
+                        "num_decode_tokens",
+                        "session_id",
+                        "block_hash_ids",
+                    ),
+                )
+                writer.writeheader()
+                writer.writerow(
+                    {
+                        "arrived_at": 0,
+                        "num_prefill_tokens": 7,
+                        "num_decode_tokens": 1,
+                        "session_id": 1,
+                        "block_hash_ids": "[]",
+                    }
+                )
+                writer.writerow(
+                    {
+                        "arrived_at": 1,
+                        "num_prefill_tokens": 16,
+                        "num_decode_tokens": 2,
+                        "session_id": 2,
+                        "block_hash_ids": 123,
+                    }
+                )
+            metrics = root / "metrics" / "cell"
+            metrics.mkdir(parents=True)
+            with (metrics / "request_metrics.csv").open("w", newline="") as stream:
+                writer = csv.DictWriter(
+                    stream,
+                    fieldnames=(
+                        "request_waiting_time_total",
+                        "ttft",
+                        "tpot",
+                        "request_e2e_time",
+                        "request_cached_prefill_tokens",
+                        "request_prefix_cache_query_blocks",
+                        "request_prefix_cache_hit_blocks",
+                    ),
+                )
+                writer.writeheader()
+                writer.writerow(
+                    {
+                        "request_waiting_time_total": 0,
+                        "ttft": 10,
+                        "tpot": "",
+                        "request_e2e_time": 10,
+                        "request_cached_prefill_tokens": 0,
+                        "request_prefix_cache_query_blocks": 0,
+                        "request_prefix_cache_hit_blocks": 0,
+                    }
+                )
+                writer.writerow(
+                    {
+                        "request_waiting_time_total": 0,
+                        "ttft": 20,
+                        "tpot": 5,
+                        "request_e2e_time": 25,
+                        "request_cached_prefill_tokens": 0,
+                        "request_prefix_cache_query_blocks": 1,
+                        "request_prefix_cache_hit_blocks": 0,
+                    }
+                )
+            (metrics / "system_metrics.json").write_text("{}")
+            (metrics / "frontier_stage_batch_ledger.jsonl").write_text(
+                json.dumps({"request_num_tokens": [1]}) + "\n"
+            )
+            summary = module.summarize(trace, root / "metrics", 2)
+
+        self.assertEqual(summary["requests"], 2)
+        self.assertEqual(summary["latency_ms"]["tpot"]["count"], 1)
+        self.assertEqual(summary["latency_ms"]["tpot"]["mean"], 5)
+
+
+if __name__ == "__main__":
+    unittest.main()

runs/frontier-collective-joint-v0/run_frontier_with_curves.py

@@ -0,0 +1,192 @@
+#!/usr/bin/env python3
+"""Run Frontier with pure-decode serving-path MoE and/or collective curves."""
+
+from __future__ import annotations
+
+import atexit
+import csv
+import json
+import os
+from collections import Counter
+from pathlib import Path
+
+
+csv.field_size_limit(16 * 1024 * 1024)
+
+
+COLLECTIVE_PAYLOAD = json.loads(
+    Path(os.environ["FRONTIER_COLLECTIVE_CURVE"]).read_text()
+)
+COLLECTIVE_VARIANT = os.environ.get("FRONTIER_COLLECTIVE_CURVE_VARIANT", "drop_mean")
+if "curve_variants_ms_per_step" in COLLECTIVE_PAYLOAD:
+    try:
+        COLLECTIVE_CURVE = COLLECTIVE_PAYLOAD["curve_variants_ms_per_step"][
+            COLLECTIVE_VARIANT
+        ]
+    except KeyError as error:
+        raise ValueError(
+            f"collective curve has no variant {COLLECTIVE_VARIANT!r}"
+        ) from error
+else:
+    if COLLECTIVE_VARIANT != "drop_mean":
+        raise ValueError("legacy collective curve only supports drop_mean")
+    COLLECTIVE_CURVE = COLLECTIVE_PAYLOAD.get(
+        "curve_ms_per_step", COLLECTIVE_PAYLOAD
+    )
+MOE_CURVE_PATH = os.environ.get("FRONTIER_FUSED_MOE_CURVE")
+MOE_CURVE = (
+    json.loads(Path(MOE_CURVE_PATH).read_text()) if MOE_CURVE_PATH else None
+)
+USAGE_PATH = Path(os.environ["FRONTIER_CURVE_USAGE"])
+USAGE: Counter[str] = Counter()
+
+
+def write_usage() -> None:
+    USAGE_PATH.parent.mkdir(parents=True, exist_ok=True)
+    USAGE_PATH.write_text(json.dumps(dict(sorted(USAGE.items())), indent=2) + "\n")
+
+
+atexit.register(write_usage)
+
+from frontier.execution_time_predictor.sklearn_moe_execution_time_predictor import (  # noqa: E402
+    SklearnMoEExecutionTimePredictor,
+)
+
+
+_ORIGINAL_ATTN_COLLECTIVE = (
+    SklearnMoEExecutionTimePredictor._get_tensor_parallel_communication_time
+)
+_ORIGINAL_MOE_COLLECTIVE = (
+    SklearnMoEExecutionTimePredictor._get_moe_tensor_parallel_allreduce_time
+)
+_ORIGINAL_GROUPED_GEMM = SklearnMoEExecutionTimePredictor._get_grouped_gemm_time
+_ORIGINAL_ATTN_NORM = (
+    SklearnMoEExecutionTimePredictor._get_attn_norm_layer_act_execution_time
+)
+_ORIGINAL_MLP_NORM = (
+    SklearnMoEExecutionTimePredictor._get_mlp_norm_layer_act_execution_time
+)
+
+
+def _pure_decode_point(self, batch) -> tuple[str, str] | None:
+    if batch is None or not bool(getattr(batch, "is_pure_decode_batch", False)):
+        return None
+    tp = str(int(self._replica_config.moe_tensor_parallel_size))
+    attn_tp = str(int(self._replica_config.attn_tensor_parallel_size))
+    if tp != attn_tp:
+        raise ValueError(
+            "Serving collective curve requires equal attention/MoE TP, got "
+            f"attn_tp={attn_tp}, moe_tp={tp}"
+        )
+    return tp, str(len(batch.requests))
+
+
+def _collective_path_time(self, batch, path: str) -> float:
+    point = _pure_decode_point(self, batch)
+    if point is None:
+        original = (
+            _ORIGINAL_ATTN_COLLECTIVE
+            if path == "attention"
+            else _ORIGINAL_MOE_COLLECTIVE
+        )
+        return original(self, batch)
+    tp, decode_batch = point
+    if tp == "1":
+        original = (
+            _ORIGINAL_ATTN_COLLECTIVE
+            if path == "attention"
+            else _ORIGINAL_MOE_COLLECTIVE
+        )
+        value = original(self, batch)
+        if value != 0.0:
+            raise ValueError(f"TP1 {path} collective must be zero, got {value}")
+        USAGE[f"collective:{path}:tp1-b{decode_batch}:structural-zero"] += 1
+        return value
+    if tp not in COLLECTIVE_CURVE or decode_batch not in COLLECTIVE_CURVE[tp]:
+        raise ValueError(
+            "Collective curve has no exact pure-decode point for "
+            f"TP={tp}, batch={decode_batch}; refusing to extrapolate"
+        )
+    layers = int(self._num_layers_per_pipeline_stage)
+    if layers <= 0:
+        raise ValueError(f"invalid layers per pipeline stage: {layers}")
+    USAGE[f"collective:{path}:tp{tp}-b{decode_batch}"] += 1
+    # The serving curve is the sum of attention and MoE output reductions over
+    # the full model step. ExecutionTime later multiplies each per-layer path.
+    return float(COLLECTIVE_CURVE[tp][decode_batch]) / (2 * layers)
+
+
+def _attention_collective_from_curve(self, batch) -> float:
+    return _collective_path_time(self, batch, "attention")
+
+
+def _moe_collective_from_curve(self, batch) -> float:
+    return _collective_path_time(self, batch, "moe")
+
+
+def _grouped_gemm_from_curve(self, num_tokens_or_allocation, batch=None) -> float:
+    if MOE_CURVE is None:
+        return _ORIGINAL_GROUPED_GEMM(
+            self, num_tokens_or_allocation, batch=batch
+        )
+    point = _pure_decode_point(self, batch)
+    if point is None:
+        return _ORIGINAL_GROUPED_GEMM(
+            self, num_tokens_or_allocation, batch=batch
+        )
+    tp, decode_batch = point
+    if tp not in MOE_CURVE or decode_batch not in MOE_CURVE[tp]:
+        raise ValueError(
+            "Fused MoE curve has no exact pure-decode point for "
+            f"TP={tp}, batch={decode_batch}; refusing to extrapolate"
+        )
+    layers = int(self._num_layers_per_pipeline_stage)
+    if layers <= 0:
+        raise ValueError(f"invalid layers per pipeline stage: {layers}")
+    USAGE[f"moe:tp{tp}-b{decode_batch}"] += 1
+    return float(MOE_CURVE[tp][decode_batch]) / layers
+
+
+def _norm_without_fused_duplicate(self, batch, *, name: str) -> float:
+    original = _ORIGINAL_ATTN_NORM if name == "attn" else _ORIGINAL_MLP_NORM
+    value = original(self, batch)
+    point = _pure_decode_point(self, batch)
+    if point is None or point[0] == "1":
+        return value
+    if value < 0:
+        raise ValueError(f"negative {name} norm prediction cannot be deducted: {value}")
+    tp, decode_batch = point
+    USAGE[f"fused_norm_deduction:{name}:tp{tp}-b{decode_batch}"] += 1
+    # AllReduceFusionPattern 1 already contains residual add + RMSNorm. Returning
+    # zero removes the exact predictor value that would otherwise be emitted in
+    # the frozen Frontier attn_norm_time/mlp_norm_time ledger row.
+    return 0.0
+
+
+def _attn_norm_without_fused_duplicate(self, batch) -> float:
+    return _norm_without_fused_duplicate(self, batch, name="attn")
+
+
+def _mlp_norm_without_fused_duplicate(self, batch) -> float:
+    return _norm_without_fused_duplicate(self, batch, name="mlp")
+
+
+SklearnMoEExecutionTimePredictor._get_tensor_parallel_communication_time = (
+    _attention_collective_from_curve
+)
+SklearnMoEExecutionTimePredictor._get_moe_tensor_parallel_allreduce_time = (
+    _moe_collective_from_curve
+)
+SklearnMoEExecutionTimePredictor._get_grouped_gemm_time = _grouped_gemm_from_curve
+SklearnMoEExecutionTimePredictor._get_attn_norm_layer_act_execution_time = (
+    _attn_norm_without_fused_duplicate
+)
+SklearnMoEExecutionTimePredictor._get_mlp_norm_layer_act_execution_time = (
+    _mlp_norm_without_fused_duplicate
+)
+
+from frontier.main import main  # noqa: E402
+
+
+if __name__ == "__main__":
+    main()

runs/frontier-prefill-kvgrowth-fix-v0/build_profile_v5.py

@@ -0,0 +1,242 @@
+#!/usr/bin/env python3
+"""Build profile-v5-kvgrowth: profile-v4-trace-final + chunked-prefill KV-context grid.
+
+Appends attention rows built from the new raw flashattn-kvgrowth-tp{1,2,4}.json
+artifacts to the frozen profile-v4 attention.csv. moe/linear are copied unchanged
+(the fix targets only the attention KV-growth term; one variable at a time).
+
+Row schema replicates freeze_frontier_profiles.py's single-segment prefill path
+(mode=vllm020_batch_spec) so the Frontier predictor ingests the new rows exactly
+like the existing chunked-prefill samples.
+
+Anchor check: re-measured q1ks8k/q512s4k must agree with profile-v4 within
+ANCHOR_TOLERANCE, otherwise the profiler stack drifted and the merge aborts.
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import hashlib
+import json
+import math
+import re
+import shutil
+from pathlib import Path
+from typing import Any
+
+STAT_NAMES = ("min", "max", "mean", "median", "std")
+ATTENTION_OPS = (
+    "attn_input_reshape",
+    "attn_kv_cache_save",
+    "attn_prefill",
+    "attn_decode",
+    "attn_output_reshape",
+)
+ANCHOR_SPECS = {"q1ks8k", "q512s4k"}
+ANCHOR_TOLERANCE = 0.25  # relative diff on attn core mean vs profile-v4
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--base", type=Path, required=True,
+                        help="profile-v4-trace-final directory")
+    parser.add_argument("--raw", type=Path, nargs="+", required=True,
+                        help="flashattn-kvgrowth-tp*.json raw artifacts")
+    parser.add_argument("--output", type=Path, required=True,
+                        help="profile-v5-kvgrowth output directory")
+    parser.add_argument("--max-model-len", type=int, default=40960)
+    parser.add_argument(
+        "--manifest-schema",
+        default="frontier-profile-v5-kvgrowth-v1",
+    )
+    return parser.parse_args()
+
+
+def parse_size(value: str, suffix: str) -> int:
+    return int(value) * (1024 if suffix == "k" else 1)
+
+
+def parse_batch_spec(spec: str) -> list[tuple[int, int]]:
+    requests: list[tuple[int, int]] = []
+    pattern = re.compile(r"^(?:(\d+))?q(\d+)(k?)(?:s(\d+)(k?))?$")
+    for segment in spec.split("_"):
+        match = pattern.match(segment)
+        if match is None:
+            raise ValueError(f"invalid vLLM batch spec: {spec}")
+        count = int(match.group(1) or 1)
+        query = parse_size(match.group(2), match.group(3))
+        seq = parse_size(match.group(4), match.group(5)) if match.group(4) else query
+        requests.extend([(query, seq)] * count)
+    return requests
+
+
+def stat_columns(prefix: str, stats: dict[str, float]) -> dict[str, float]:
+    return {f"time_stats.{prefix}.{name}": stats[name] for name in STAT_NAMES}
+
+
+def zero_stat_columns(prefix: str) -> dict[str, float]:
+    return {f"time_stats.{prefix}.{name}": 0.0 for name in STAT_NAMES}
+
+
+def attention_core_stats(raw: dict[str, Any]) -> dict[str, float]:
+    # Same policy as freeze_frontier_profiles.py: benchmark exports aggregate
+    # mean, not samples; mean stands in for median.
+    return {
+        "min": 1000.0 * float(raw["min_time"]),
+        "max": 1000.0 * float(raw["max_time"]),
+        "mean": 1000.0 * float(raw["mean_time"]),
+        "median": 1000.0 * float(raw["mean_time"]),
+        "std": 1000.0 * float(raw["std_time"]),
+    }
+
+
+def kv_update_stats(raw: dict[str, Any]) -> dict[str, float]:
+    stats = raw["kv_cache_update_time"]
+    return {name: float(stats[f"{name}_ms"]) for name in STAT_NAMES}
+
+
+def build_row(raw: dict[str, Any], max_model_len: int = 40960) -> dict[str, Any]:
+    spec = raw["config"]["batch_spec"]
+    tp = int(raw["tensor_parallel_size"])
+    requests = parse_batch_spec(spec)
+    if any(q == 1 for q, _ in requests):
+        raise ValueError(f"decode segment in prefill grid spec: {spec}")
+    queries = [q for q, _ in requests]
+    contexts = [kv - q for q, kv in requests]
+    avg_query = sum(queries) / len(queries)
+    variance = sum((q - avg_query) ** 2 for q in queries) / len(queries)
+    std = math.sqrt(variance)
+    avg_context = sum(contexts) / len(contexts)
+    row: dict[str, Any] = {}
+    for op in ATTENTION_OPS:
+        row.update(zero_stat_columns(op))
+    row.update(stat_columns("attn_kv_cache_save", kv_update_stats(raw)))
+    row.update(stat_columns("attn_prefill", attention_core_stats(raw)))
+    row.update(
+        {
+            "n_embd": 2048,
+            "n_q_head": 32,
+            "n_kv_head": 4,
+            "block_size": 16,
+            "num_tensor_parallel_workers": tp,
+            "max_model_len": max_model_len,
+            "batch_size": len(requests),
+            "prefill_chunk_size": sum(queries),
+            "kv_cache_size": avg_context,
+            "is_prefill": True,
+            "attention_backend": "FLASH_ATTN",
+            "is_mixed_batch": False,
+            "mode": "vllm020_batch_spec",
+            "seq_lens": json.dumps(queries),
+            "total_tokens": sum(queries),
+            "max_seq_len": max(queries),
+            "min_seq_len": min(queries),
+            "avg_seq_len": avg_query,
+            "equal_seq_len": len(set(queries)) == 1,
+            "seq_len_variance": variance,
+            "seq_len_std": std,
+            "seq_len_cv": std / avg_query if avg_query else 0.0,
+            "is_chunked_prefill_sample": avg_context > 0,
+            "chunk_start_token": avg_context,
+            "chunk_end_token": avg_context + sum(queries),
+            "total_prefill_tokens": sum(queries),
+            "profiling_precision": "BF16",
+            "model_arch": "generic",
+            "quant_signature": "none",
+            "measurement_type": "CUDA_EVENT",
+            "is_true_mixed_batch": False,
+            "prefill_seq_lens": "",
+            "prefill_kv_cache_sizes": "",
+            "decode_kv_cache_sizes": "",
+            "num_prefill_seqs": "",
+            "num_decode_seqs": "",
+            "decode_batch_size": "",
+            "total_batch_size": "",
+            "total_decode_tokens": "",
+            "decode_avg_kv_cache_size": "",
+            "batch_spec": spec,
+            "projection_policy": (
+                "measured_FA3_core_plus_measured_KV;reshape_assumed_zero;"
+                "mean_as_median"
+            ),
+        }
+    )
+    return row
+
+
+def check_anchor(row: dict[str, Any], base_rows: list[dict[str, str]]) -> str:
+    tp = str(row["num_tensor_parallel_workers"])
+    spec = row["batch_spec"]
+    matches = [
+        r for r in base_rows
+        if r["batch_spec"] == spec and r["num_tensor_parallel_workers"] == tp
+    ]
+    if not matches:
+        return f"anchor {spec}/TP{tp}: no profile-v4 row (skipped)"
+    old = float(matches[0]["time_stats.attn_prefill.mean"])
+    new = float(row["time_stats.attn_prefill.mean"])
+    rel = abs(new - old) / old if old else float("inf")
+    verdict = f"anchor {spec}/TP{tp}: v4={old:.4f}ms new={new:.4f}ms rel_diff={rel:.1%}"
+    if rel > ANCHOR_TOLERANCE:
+        raise SystemExit(f"PROFILER DRIFT: {verdict} exceeds {ANCHOR_TOLERANCE:.0%}")
+    return verdict
+
+
+def sha256(path: Path) -> str:
+    return hashlib.sha256(path.read_bytes()).hexdigest()
+
+
+def main() -> None:
+    args = parse_args()
+    base_csv = args.base / "attention.csv"
+    with base_csv.open(newline="") as stream:
+        reader = csv.DictReader(stream)
+        fields = list(reader.fieldnames or [])
+        base_rows = list(reader)
+
+    new_rows: list[dict[str, Any]] = []
+    anchors: list[str] = []
+    seen: set[tuple[str, str]] = set()
+    for raw_path in args.raw:
+        raw_payload = json.loads(raw_path.read_text())
+        for raw in raw_payload["rows"]:
+            if raw.get("error"):
+                raise SystemExit(f"{raw_path}: failed sample {raw['config']['batch_spec']}")
+            row = build_row(raw, args.max_model_len)
+            key = (row["batch_spec"], str(row["num_tensor_parallel_workers"]))
+            if key in seen:
+                raise SystemExit(f"duplicate sample {key} across raw inputs")
+            seen.add(key)
+            if row["batch_spec"] in ANCHOR_SPECS:
+                anchors.append(check_anchor(row, base_rows))
+            else:
+                new_rows.append(row)
+
+    args.output.mkdir(parents=True, exist_ok=True)
+    out_csv = args.output / "attention.csv"
+    with out_csv.open("w", newline="") as stream:
+        writer = csv.DictWriter(stream, fieldnames=fields)
+        writer.writeheader()
+        writer.writerows(base_rows)
+        for row in new_rows:
+            writer.writerow({field: row.get(field, "") for field in fields})
+    for name in ("moe.csv", "linear_op.csv"):
+        shutil.copy2(args.base / name, args.output / name)
+
+    manifest = {
+        "schema": args.manifest_schema,
+        "base": str(base_csv),
+        "base_sha256": sha256(base_csv),
+        "raw_inputs": {str(p): sha256(p) for p in args.raw},
+        "appended_rows": len(new_rows),
+        "max_model_len": args.max_model_len,
+        "anchor_checks": anchors,
+        "output_sha256": sha256(out_csv),
+    }
+    (args.output / "manifest.json").write_text(json.dumps(manifest, indent=2))
+    print(json.dumps(manifest, indent=2))
+
+
+if __name__ == "__main__":
+    main()

runs/frontier-prefill-kvgrowth-fix-v0/profiles/profile-v5-kvgrowth/attention.csv

@@ -0,0 +1,244 @@
+time_stats.attn_input_reshape.min,time_stats.attn_input_reshape.max,time_stats.attn_input_reshape.mean,time_stats.attn_input_reshape.median,time_stats.attn_input_reshape.std,time_stats.attn_kv_cache_save.min,time_stats.attn_kv_cache_save.max,time_stats.attn_kv_cache_save.mean,time_stats.attn_kv_cache_save.median,time_stats.attn_kv_cache_save.std,time_stats.attn_prefill.min,time_stats.attn_prefill.max,time_stats.attn_prefill.mean,time_stats.attn_prefill.median,time_stats.attn_prefill.std,time_stats.attn_decode.min,time_stats.attn_decode.max,time_stats.attn_decode.mean,time_stats.attn_decode.median,time_stats.attn_decode.std,time_stats.attn_output_reshape.min,time_stats.attn_output_reshape.max,time_stats.attn_output_reshape.mean,time_stats.attn_output_reshape.median,time_stats.attn_output_reshape.std,n_embd,n_q_head,n_kv_head,block_size,num_tensor_parallel_workers,max_model_len,batch_size,prefill_chunk_size,kv_cache_size,is_prefill,attention_backend,is_mixed_batch,mode,seq_lens,total_tokens,max_seq_len,min_seq_len,avg_seq_len,equal_seq_len,seq_len_variance,seq_len_std,seq_len_cv,is_chunked_prefill_sample,chunk_start_token,chunk_end_token,total_prefill_tokens,profiling_precision,model_arch,quant_signature,measurement_type,is_true_mixed_batch,prefill_seq_lens,prefill_kv_cache_sizes,decode_kv_cache_sizes,num_prefill_seqs,num_decode_seqs,decode_batch_size,total_batch_size,total_decode_tokens,decode_avg_kv_cache_size,batch_composition_ratio,batch_spec,projection_policy
+0.0,0.0,0.0,0.0,0.0,0.01414399966597557,0.028863999992609024,0.019705599918961526,0.01771199982613325,0.005157200849836681,0.047968000173568726,0.07046400010585785,0.05810240097343922,0.05810240097343922,0.007477463486041561,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,64,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,False,0.0,64.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.04947200044989586,0.020412799902260303,0.01635199971497059,0.010107497379722417,0.046560000628232956,0.08323200047016144,0.05587520003318787,0.05587520003318787,0.011126758739503428,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,128,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,False,0.0,128.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.022207999601960182,0.017033600155264138,0.015312000177800655,0.002819235991970241,0.05104000121355057,0.07692799717187881,0.056396800279617305,0.056396800279617305,0.007481982178637539,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,256,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,False,0.0,256.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015072000212967396,0.022272000089287758,0.01706880023702979,0.01616000011563301,0.002460889579319197,0.06931199878454208,0.0838719978928566,0.07432000041007995,0.07432000041007995,0.004777766433175866,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,512,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,False,0.0,512.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.018592000007629395,0.028543999418616295,0.02095999978482723,0.019183999858796597,0.003198175496053494,0.12179200351238251,0.15408000349998474,0.1307712011039257,0.1307712011039257,0.00858807797538298,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.027775999158620834,0.03385600075125694,0.030131200328469276,0.029680000618100166,0.0021152558103575215,0.32678401470184326,0.3450239896774292,0.33396480381488797,0.33396480381488797,0.0045872424917606375,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04438399896025658,0.05084799975156784,0.046540799736976626,0.04531199857592583,0.002277905811237223,1.0959680080413818,1.1151360273361206,1.0999775886535645,1.0999775886535645,0.005694403246120485,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.078015998005867,0.08691199868917465,0.08114239946007729,0.08019199967384338,0.00292795706334475,4.070400238037109,4.113152027130127,4.087088012695312,4.087088012695312,0.013660567012509554,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016063999384641647,0.05167999863624573,0.022115200012922286,0.017583999782800674,0.010340094822340818,0.05196800082921982,0.09011200070381165,0.06328320093452933,0.06328320093452933,0.012557341255467452,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,64,448.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,True,448.0,512.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64s512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01583999954164028,0.026623999699950218,0.018927999772131443,0.017376000061631203,0.003514650316260619,0.06681600213050842,0.07993599772453308,0.0725280001759529,0.0725280001759529,0.004343502558613716,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,128,896.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,True,896.0,1024.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01648000068962574,0.030880000442266464,0.01945280022919178,0.017967999912798405,0.004096211183007485,0.1311360001564026,0.1546880006790161,0.13908160030841826,0.13908160030841826,0.007511906874366178,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,256,1792.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,True,1792.0,2048.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.017855999991297722,0.03558399900794029,0.020851199887692927,0.018559999763965607,0.005235911594130716,0.32950401306152344,0.350271999835968,0.33912960588932034,0.33912960588932034,0.006027400986663648,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,512,3584.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,True,3584.0,4096.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.019328000023961067,0.040608000010252,0.022790400311350822,0.020704000256955624,0.006113051965778337,1.1415679454803467,1.1518720388412476,1.144483208656311,1.144483208656311,0.0032332311374389127,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,1024,7168.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,True,7168.0,8192.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1ks8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015807999297976494,0.030688000842928886,0.019596799835562707,0.01774400006979704,0.004343384771033462,0.0,0.0,0.0,0.0,0.0,0.049056001007556915,0.07580800354480743,0.05948160067200661,0.05948160067200661,0.009031541471446955,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01603199914097786,0.02486399933695793,0.01923839971423149,0.018079999834299088,0.0032282528537266424,0.0,0.0,0.0,0.0,0.0,0.05142400041222572,0.07353600114583969,0.059328000620007516,0.059328000620007516,0.0073307807735143084,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016543999314308167,0.03977600112557411,0.021379199624061585,0.018511999398469925,0.006593576176246171,0.0,0.0,0.0,0.0,0.0,0.0488319993019104,0.06435199826955795,0.05479039996862411,0.05479039996862411,0.005672522998491864,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01635199971497059,0.02844800055027008,0.019267200119793416,0.017952000722289085,0.0035068687666949577,0.0,0.0,0.0,0.0,0.0,0.049855999648571014,0.07798399776220322,0.05986879989504815,0.05986879989504815,0.01043914754878828,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016383999958634377,0.026079999282956123,0.01923519968986511,0.017791999503970146,0.0032161974331284568,0.0,0.0,0.0,0.0,0.0,0.058111999183893204,0.1045759990811348,0.06708480007946492,0.06708480007946492,0.013479022462646494,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.04057599976658821,0.019100800156593323,0.015455999877303839,0.007512281243011577,0.0,0.0,0.0,0.0,0.0,0.05363199859857559,0.07782399654388428,0.06090559959411622,0.06090559959411622,0.007544620176348091,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014431999996304512,0.02191999927163124,0.016684799920767546,0.01563199982047081,0.0024293118621811216,0.0,0.0,0.0,0.0,0.0,0.0629120022058487,0.07891199737787247,0.06891520097851753,0.06891520097851753,0.005472695665695425,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014560000039637089,0.038943998515605927,0.018313600029796363,0.01561600062996149,0.007127270260115769,0.0,0.0,0.0,0.0,0.0,0.08675199747085571,0.10662399977445602,0.09391999915242194,0.09391999915242194,0.006988099589086635,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.054687999188899994,0.021439999900758268,0.01539199985563755,0.012052764849597775,0.0,0.0,0.0,0.0,0.0,0.13488000631332397,0.1528639942407608,0.1431359991431236,0.1431359991431236,0.005436271464033599,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.041919998824596405,0.01899839974939823,0.015343999955803156,0.007989843526623287,0.0,0.0,0.0,0.0,0.0,0.06176000088453293,0.08374399691820145,0.06747519969940186,0.06747519969940186,0.0066067747128778,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016256000846624374,0.11353600025177002,0.042761600017547606,0.028960000723600388,0.029104301538020762,0.0,0.0,0.0,0.0,0.0,0.09734400361776352,0.14422400295734406,0.11392960175871848,0.11392960175871848,0.013198594600417867,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.034143999218940735,0.018918400071561335,0.01643200032413006,0.005500943993080684,0.0,0.0,0.0,0.0,0.0,0.12918399274349213,0.15087999403476715,0.13807999789714814,0.13807999789714814,0.007658330538677587,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016063999384641647,0.03641600161790848,0.0198208000510931,0.01780799962580204,0.0057264128169845765,0.0,0.0,0.0,0.0,0.0,0.22099199891090393,0.23904000222682953,0.2293503984808922,0.2293503984808922,0.004861342907006028,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014751999638974667,0.035840000957250595,0.018908800091594458,0.015792000107467175,0.0064374817924757475,0.0,0.0,0.0,0.0,0.0,0.10134399682283401,0.12201599776744843,0.10896319895982742,0.10896319895982742,0.006336330809165179,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013856000266969204,0.03417599946260452,0.017846399918198586,0.014800000004470348,0.006495007539635255,0.0,0.0,0.0,0.0,0.0,0.13126400113105774,0.15561600029468536,0.1389280006289482,0.1389280006289482,0.008381472811075022,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014431999996304512,0.03519999980926514,0.019168000388890504,0.01600000075995922,0.005995522477654695,0.0,0.0,0.0,0.0,0.0,0.21728000044822693,0.2343679964542389,0.2231455981731415,0.2231455981731415,0.004720730646739123,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014047999866306782,0.03670400008559227,0.018441599886864425,0.015023999847471714,0.006596535162793127,0.0,0.0,0.0,0.0,0.0,0.39321601390838623,0.4524799883365631,0.4058080047369003,0.4058080047369003,0.01578349755088941,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014336000196635723,0.022143999114632607,0.016912000067532063,0.015168000012636185,0.0028156089295136347,0.0,0.0,0.0,0.0,0.0,0.15587200224399567,0.3079040050506592,0.17838079929351805,0.17838079929351805,0.04355865575265927,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.02112000063061714,0.015516800060868263,0.01488000014796853,0.001940870731593904,0.0,0.0,0.0,0.0,0.0,0.21587200462818146,0.23561599850654602,0.22250880002975468,0.22250880002975468,0.006181951170646666,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015552000142633915,0.039264000952243805,0.023609600123018028,0.02131200022995472,0.007236979625711548,0.0,0.0,0.0,0.0,0.0,0.408735990524292,0.470335990190506,0.4336863994598388,0.4336863994598388,0.01844662383160074,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.025407999753952026,0.016227199975401164,0.014960000291466713,0.0031617375441736185,0.0,0.0,0.0,0.0,0.0,0.7412800192832947,0.7627840042114258,0.7464000046253203,0.7464000046253203,0.006112167448837547,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01462399959564209,0.02812799997627735,0.020652799773961304,0.021359999664127827,0.004308706957613102,0.028383498565450627,0.039859687970646644,0.032492258074592426,0.032492258074592426,0.00453597266208597,0.029312501176103633,0.04116431058787463,0.03355574193327539,0.03355574193327539,0.004684436757701648,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,72,,,,,,,,False,,,64,BF16,generic,none,CUDA_EVENT,True,[64],[0],"[512, 512, 512, 512, 512, 512, 512, 512]",1,8,8,9,8,512.0,0.1111111111111111,q64_8q1s512,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.015552000142633915,0.034143999218940735,0.023171199765056372,0.024255999363958836,0.005908614918096971,0.03333159243114438,0.038935341782478095,0.03580428402241854,0.03580428402241854,0.002082270297044095,0.03633240903369937,0.04244065945634484,0.03902771507087562,0.03902771507087562,0.0022697354261490147,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,136,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,8,8,9,8,1024.0,0.1111111111111111,q128_8q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.02611199952661991,0.019635199941694735,0.018112000077962875,0.0038298050749257795,0.04189529417991216,0.057484239920526384,0.04744885718421094,0.04744885718421094,0.004779830748455743,0.051672703037266184,0.07089975418195164,0.05852234134479412,0.05852234134479412,0.005895334539786378,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,144,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,16,16,17,16,1024.0,0.058823529411764705,q128_16q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.029311999678611755,0.018662399891763926,0.01673599984496832,0.0044017112162725355,0.04322973959325901,0.05049827064705393,0.04535414343408448,0.04535414343408448,0.0022944156000240697,0.08733025617719538,0.10201372836398578,0.09162185574241775,0.09162185574241775,0.004635047631846023,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,272,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,16,16,17,16,2048.0,0.058823529411764705,q256_16q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014592000283300877,0.026367999613285065,0.016336000058799982,0.01515199989080429,0.003415353455946402,0.06031842775160765,0.06618323188375198,0.06274415549817247,0.06274415549817247,0.001985647289644255,0.14768157653992678,0.16204076249052324,0.15362064543185072,0.15362064543185072,0.004861590945216247,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,288,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,32,32,33,32,2048.0,0.030303030303030304,q256_32q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014751999638974667,0.02454400062561035,0.017167999967932702,0.016191999427974224,0.0028685016454498436,0.09128700688359712,0.09689150775996329,0.09360396051475776,0.09360396051475776,0.0013477542744916764,0.2740889887523892,0.2909164873209846,0.2810456356995366,0.2810456356995366,0.004046628526808561,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,544,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q512_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01881599985063076,0.03097599931061268,0.024598400108516216,0.024848000146448612,0.0037539437391565975,0.1035249255866932,0.10603132147437412,0.10399797220840973,0.10399797220840973,0.0007025109429056814,0.5652750707893444,0.5789606938874117,0.5678580377517648,0.5678580377517648,0.003835906384194897,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,576,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,64,64,65,64,4096.0,0.015384615384615385,q512_64q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018624000251293182,0.043487999588251114,0.024460799992084503,0.019600000232458115,0.008922081629781394,0.196169204945307,0.2076140047945799,0.2008444429250931,0.2008444429250931,0.00394350953292805,1.1196708009293268,1.1849940417370974,1.1463555573610091,1.1463555573610091,0.022508285530529783,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192]",1,64,64,65,64,8192.0,0.015384615384615385,q1k_64q1s8k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.027135999873280525,0.03667199984192848,0.02945920005440712,0.028256000019609928,0.0029446678408169553,0.37757279619664613,0.3898113624476372,0.38075520430942184,0.38075520430942184,0.0036600203831042254,0.2522831942132049,0.2604606493092597,0.25440958703617433,0.25440958703617433,0.0024455194930253126,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q2k_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.0435199998319149,0.049695998430252075,0.04502719938755036,0.04395199939608574,0.002035785660169308,1.1048984388245497,1.1189053886476108,1.1112371236754128,1.1112371236754128,0.0050220696724624985,0.1395495077239122,0.14131859607071193,0.14035008841031085,0.14035008841031085,0.0006342911944856293,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,16,16,17,16,4096.0,0.058823529411764705,q4k_16q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018783999606966972,0.024288000538945198,0.020627199858427047,0.019567999988794327,0.0019457052717059358,0.09455999732017517,0.12185599654912949,0.10207359939813614,0.10207359939813614,0.00753346544014261,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,2,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512]",1024,512,512,512.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.027295999228954315,0.034752000123262405,0.02943360023200512,0.028528000228106976,0.0023159160681123767,0.14416000247001648,0.15904000401496887,0.14979200065135959,0.14979200065135959,0.00480624675866005,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,4,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512]",2048,512,512,512.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.043455999344587326,0.05084799975156784,0.045500800386071204,0.04411200061440468,0.002490556062831049,0.24454399943351746,0.25865599513053894,0.2504959970712662,0.2504959970712662,0.003778494140301353,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512]",4096,512,512,512.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07673600316047668,0.08374399691820145,0.07970559895038605,0.07873599976301193,0.0022907976135004057,0.445248007774353,0.4758400022983551,0.45409599840641024,0.45409599840641024,0.009133230340383306,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512]",8192,512,512,512.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.043296001851558685,0.051072001457214355,0.044972800090909,0.04399999976158142,0.002451216824441962,0.6147199869155884,0.6290879845619202,0.6204927921295166,0.6204927921295166,0.004344846739788608,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,2,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048]",4096,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07737600058317184,0.09123200178146362,0.08094720020890236,0.07980800047516823,0.004065264798368611,1.1744320392608643,1.1887680292129517,1.178323209285736,1.178323209285736,0.004395876469319665,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,4,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048, 2048, 2048]",8192,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.03254399821162224,0.01810879958793521,0.01611199975013733,0.005058478889747962,0.0,0.0,0.0,0.0,0.0,0.06339199841022491,0.0841279998421669,0.07019519805908202,0.07019519805908202,0.006070126182471763,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.030271999537944794,0.018124799989163876,0.015711999498307705,0.004696400814632773,0.0,0.0,0.0,0.0,0.0,0.2739199995994568,0.2922239899635315,0.28281279802322384,0.28281279802322384,0.00633037978599564,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.022431999444961548,0.016844799742102623,0.015471999999135733,0.0027624803153841917,0.0,0.0,0.0,0.0,0.0,0.3909119963645935,0.4160960018634796,0.39996159672737125,0.39996159672737125,0.007037174636804898,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014240000396966934,0.035071998834609985,0.018764800019562246,0.015583999920636415,0.006305594700523817,0.0,0.0,0.0,0.0,0.0,0.7383679747581482,0.7597119808197021,0.7430047929286957,0.7430047929286957,0.006192645960911466,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.028991999104619026,0.019401599932461978,0.01665600063279271,0.005434953793780546,0.0,0.0,0.0,0.0,0.0,1.427008032798767,1.4517120122909546,1.4361984014511109,1.4361984014511109,0.008802755821028187,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.02844800055027008,0.01820160010829568,0.014944000169634819,0.004942002036909087,0.0,0.0,0.0,0.0,0.0,0.08361600339412689,0.11027199774980545,0.09058240056037903,0.09058240056037903,0.00872938604423622,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014112000353634357,0.023104000836610794,0.01648960020393133,0.015632000286132097,0.0027628828340349578,0.0,0.0,0.0,0.0,0.0,0.5063040256500244,0.5497599840164185,0.5168287932872773,0.5168287932872773,0.013557229606313293,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01616000011563301,0.029055999591946602,0.018780799955129622,0.017215999774634838,0.003794434947431941,0.0,0.0,0.0,0.0,0.0,0.7439360022544861,0.7719680070877075,0.7502080142498017,0.7502080142498017,0.00839205598262567,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015968000516295433,0.03046399913728237,0.018822400271892546,0.01726400014013052,0.004212536831927695,0.0,0.0,0.0,0.0,0.0,1.4256000518798828,1.449504017829895,1.4325888037681578,1.4325888037681578,0.0065448028108711165,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.027712000533938408,0.01633920017629862,0.014944000169634819,0.0038488220071983326,0.0,0.0,0.0,0.0,0.0,2.798719882965088,3.0184640884399414,2.8293471813201903,2.8293471813201903,0.06375662767704417,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.021407999098300934,0.01637439979240298,0.014992000069469213,0.0024463594774459265,0.0,0.0,0.0,0.0,0.0,0.09347199648618698,0.10473600029945374,0.09820479974150656,0.09820479974150656,0.004061668684999473,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014303999952971935,0.04790399968624115,0.020179199893027543,0.016447999514639378,0.009635010283506967,0.0,0.0,0.0,0.0,0.0,0.6221439838409424,0.6444799900054932,0.6277984082698822,0.6277984082698822,0.007234140179397069,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,8,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.03276799991726875,0.01921279989182949,0.015232000034302473,0.0065506148511265076,0.0,0.0,0.0,0.0,0.0,0.9111359715461731,0.9307839870452881,0.9151648044586183,0.9151648044586183,0.005528712062927265,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,16,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014175999909639359,0.02707199938595295,0.016512000095099212,0.014688000082969666,0.0038989080209321414,0.0,0.0,0.0,0.0,0.0,1.7645119428634644,1.7849279642105103,1.7684095859527589,1.7684095859527589,0.005876963340184193,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,32,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014240000396966934,0.029184000566601753,0.016825600154697896,0.014752000104635954,0.00457910514148248,0.0,0.0,0.0,0.0,0.0,3.4781761169433594,3.5388801097869873,3.490892815589905,3.490892815589905,0.01818910600692522,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,64,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014976000413298607,0.03139200061559677,0.01898880014196038,0.016848000697791576,0.005083715524393418,0.13752702814163098,0.14117629917511842,0.13848196486571557,0.13848196486571557,0.0009880564964713527,0.523336967410432,0.5372236808930884,0.5269708253945994,0.5269708253945994,0.00375988994658546,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,520,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,8,8,9,8,16384.0,0.1111111111111111,q512_8q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01833599992096424,0.02707199938595295,0.020995199866592883,0.01976000051945448,0.0028392656352050185,0.17817885890237703,0.18622915300900206,0.18109068484526028,0.18109068484526028,0.0027040006080457624,0.5449571488834201,0.5695788427872015,0.5538629212834322,0.5538629212834322,0.008270141985514729,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1040,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,16,16,17,16,16384.0,0.058823529411764705,q1k_16q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.02703999914228916,0.03308799862861633,0.028883199393749236,0.02759999968111515,0.002285758578932753,0.4688266550410815,0.48301665772804186,0.4727750380198454,0.4727750380198454,0.004396396389258256,1.0430453981052807,1.0746153117238624,1.051829759343436,1.051829759343436,0.009781101336187197,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,32,32,33,32,16384.0,0.030303030303030304,q2k_32q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.04368000105023384,0.059328000992536545,0.04809600040316582,0.04617599956691265,0.005158792915002645,1.3509461459747514,1.3759066409666496,1.3589365122072,1.3589365122072,0.008008877716183384,0.9213738861449998,0.9383974724214122,0.9268234851606594,0.9268234851606594,0.005462224239661061,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,16,16,17,16,32768.0,0.058823529411764705,q4k_16q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.026944000273942947,0.040511999279260635,0.030291200056672095,0.02817599941045046,0.004386386489305873,0.5071830964059985,0.5142792136001758,0.5096392092471257,0.5096392092471257,0.0022059272685869546,2.175632932188972,2.2060727208328075,2.186168772243963,2.186168772243963,0.009462633998570079,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,32,32,33,32,32768.0,0.030303030303030304,q2k_32q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018303999677300453,0.03017600066959858,0.02039040010422468,0.018943999893963337,0.0034612051983613614,0.21341429693945616,0.21552776300295579,0.21393496609766816,0.21393496609766816,0.0005736773262009222,4.617401495144284,4.663128147488988,4.628666619290515,4.628666619290515,0.012412001359412127,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,64,64,65,64,32768.0,0.015384615384615385,q1k_64q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.02364799939095974,0.01809599995613098,0.016352000646293163,0.0035481127058959038,0.04822399839758873,0.08566399663686752,0.05961279980838299,0.05961279980838299,0.011445665413968877,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,64,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,False,0.0,64.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014175999909639359,0.020864000543951988,0.01668160008266568,0.015664000064134598,0.0025769063833097584,0.049695998430252075,0.08057600259780884,0.05882879942655563,0.05882879942655563,0.009515126108519331,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,128,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,False,0.0,128.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.028704000636935234,0.017430400010198355,0.015056000091135502,0.004349294613335555,0.049855999648571014,0.07366400212049484,0.05459520071744919,0.05459520071744919,0.0069610920757925574,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,256,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,False,0.0,256.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014336000196635723,0.03161599859595299,0.017788799852132796,0.015711999963968992,0.005005385723318659,0.06102399900555611,0.08179199695587158,0.06650560013949873,0.06650560013949873,0.006947995595801105,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,512,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,False,0.0,512.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.04416000097990036,0.019200000166893005,0.015488000120967627,0.008561241323364038,0.08054400235414505,0.09196799993515015,0.08607039973139763,0.08607039973139763,0.004145329035483754,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.017855999991297722,0.023391999304294586,0.019667199812829494,0.018463999964296818,0.0022577669687832585,0.18729600310325623,0.20585599541664124,0.19546559900045393,0.19546559900045393,0.006339068824303663,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.026240000501275063,0.03446400165557861,0.028297600522637367,0.027088000439107418,0.0025148441522922374,0.5754240155220032,0.5889919996261597,0.5800191938877105,0.5800191938877105,0.003858869273829596,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04182400181889534,0.047200001776218414,0.043036799877882004,0.0423360001295805,0.0017073405772076728,2.063199996948242,2.0787200927734375,2.067151999473572,2.067151999473572,0.004959651271127835,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.05648000165820122,0.02270399993285537,0.017935999669134617,0.012377915150727689,0.049536000937223434,0.07196799665689468,0.056015999615192415,0.056015999615192415,0.0070476637552742884,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,64,448.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,True,448.0,512.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64s512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01408000010997057,0.021344000473618507,0.01611520005390048,0.014928000047802925,0.0025499884993961702,0.07072000205516815,0.2642880082130432,0.1588256008923054,0.1588256008923054,0.053220347086102376,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,128,896.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,True,896.0,1024.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01360000018030405,0.03014400042593479,0.016975999902933837,0.015056000091135502,0.004715159697364111,0.08393599838018417,0.11036799848079681,0.09160000011324881,0.09160000011324881,0.007911669434472792,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,256,1792.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,True,1792.0,2048.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.020767999812960625,0.016233599931001663,0.015392000321298838,0.002202615907995427,0.1998399943113327,0.22070400416851044,0.20855360180139543,0.20855360180139543,0.00689307200230667,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,512,3584.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,True,3584.0,4096.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.033440001308918,0.018684800155460833,0.016048000194132328,0.00553991005639174,0.6043199896812439,0.635807991027832,0.6126143991947173,0.6126143991947173,0.008745933953408096,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,1024,7168.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,True,7168.0,8192.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1ks8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013824000023305416,0.03359999880194664,0.017811199743300678,0.014944000169634819,0.005926140483565472,0.0,0.0,0.0,0.0,0.0,0.045471999794244766,0.07539200037717819,0.054758400097489356,0.054758400097489356,0.010253548506101549,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014047999866306782,0.02409599907696247,0.01641279999166727,0.01473599998280406,0.003347158638713987,0.0,0.0,0.0,0.0,0.0,0.0461760014295578,0.07529599964618683,0.05460800044238568,0.05460800044238568,0.009748937798340135,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.028416000306606293,0.018611199874430894,0.01598400017246604,0.005209875093863647,0.0,0.0,0.0,0.0,0.0,0.048128001391887665,0.07897599786520004,0.061353600397706036,0.061353600397706036,0.010488153655157845,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014112000353634357,0.025728000327944756,0.016092800162732603,0.01512000011280179,0.003300888123052605,0.0,0.0,0.0,0.0,0.0,0.04864000156521797,0.07648000121116638,0.05810560062527656,0.05810560062527656,0.009473544218404408,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.03551999852061272,0.018588799890130757,0.016064000315964222,0.006071057437992447,0.0,0.0,0.0,0.0,0.0,0.04822399839758873,0.07862400263547897,0.05660480037331582,0.05660480037331582,0.009565394213730401,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014303999952971935,0.028831999748945236,0.01699519995599985,0.015024000313133001,0.004285000744868188,0.0,0.0,0.0,0.0,0.0,0.04854400083422661,0.06719999760389328,0.05778240002691746,0.05778240002691746,0.006852125554679805,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.0144640002399683,0.024927999824285507,0.0161183999851346,0.01521599991247058,0.0029774808524673907,0.0,0.0,0.0,0.0,0.0,0.05379199981689453,0.08966399729251862,0.06270079985260964,0.06270079985260964,0.009983591277092696,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014431999996304512,0.03001599945127964,0.017648000083863736,0.01547200046479702,0.004585126309484848,0.0,0.0,0.0,0.0,0.0,0.061664000153541565,0.07692799717187881,0.06704320013523103,0.06704320013523103,0.005500191798490629,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014175999909639359,0.026367999613285065,0.016947199776768684,0.015039999969303608,0.0037951566103550205,0.0,0.0,0.0,0.0,0.0,0.08799999952316284,0.111455999314785,0.0964031994342804,0.0964031994342804,0.007397541615558088,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01369599997997284,0.021247999742627144,0.015359999984502793,0.014640000183135271,0.0020942770950814317,0.0,0.0,0.0,0.0,0.0,0.051711998879909515,0.07065600156784058,0.058054400235414506,0.058054400235414506,0.006633034815910025,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014208000153303146,0.0307839997112751,0.017148799914866685,0.015039999969303608,0.004882374686312284,0.0,0.0,0.0,0.0,0.0,0.061919998377561576,0.07843200117349625,0.06628479920327664,0.06628479920327664,0.004962852689801192,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013887999579310417,0.02969600073993206,0.017500799987465142,0.015232000034302473,0.00467273319705314,0.0,0.0,0.0,0.0,0.0,0.08819200098514557,0.11097600311040878,0.09493440166115762,0.09493440166115762,0.007509235042985577,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.022112000733613968,0.01751680001616478,0.016047999262809753,0.0030306621792915785,0.0,0.0,0.0,0.0,0.0,0.13065600395202637,0.15110400319099426,0.13857279866933822,0.13857279866933822,0.00750137841249771,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013919999822974205,0.04012800008058548,0.01892479993402958,0.01550400024279952,0.007459545267816961,0.0,0.0,0.0,0.0,0.0,0.06278400123119354,0.08259200304746628,0.0704512007534504,0.0704512007534504,0.005979055984382744,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014336000196635723,0.02236800082027912,0.016332800220698118,0.014928000047802925,0.002784044363186731,0.0,0.0,0.0,0.0,0.0,0.1003199964761734,0.1279360055923462,0.10921279862523078,0.10921279862523078,0.00862769617046716,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013919999822974205,0.0225600004196167,0.016128000058233737,0.014800000004470348,0.002958953332547708,0.0,0.0,0.0,0.0,0.0,0.13116799294948578,0.14812800288200378,0.13783999979496003,0.13783999979496003,0.005696384361148053,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014208000153303146,0.029983999207615852,0.017468800116330386,0.01508800033479929,0.0047379550962483065,0.0,0.0,0.0,0.0,0.0,0.217631995677948,0.2447360008955002,0.22715839892625808,0.22715839892625808,0.008831828221847138,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014047999866306782,0.02304000034928322,0.016512000095099212,0.014512000139802694,0.0035026774828624254,0.0,0.0,0.0,0.0,0.0,0.11020799726247787,0.12307199835777283,0.11600959971547126,0.11600959971547126,0.004667637266950902,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014112000353634357,0.042080000042915344,0.017510399967432023,0.014607999939471483,0.00823117883530167,0.0,0.0,0.0,0.0,0.0,0.15702399611473083,0.17587199807167053,0.16399359852075576,0.16399359852075576,0.006588299074676393,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013919999822974205,0.0208320003002882,0.015299199987202883,0.01462399959564209,0.001916768086505386,0.0,0.0,0.0,0.0,0.0,0.21779200434684753,0.2415360063314438,0.2260768011212349,0.2260768011212349,0.007251352080685236,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014015999622642994,0.021088000386953354,0.015619200188666582,0.01473599998280406,0.002013227452302141,0.0,0.0,0.0,0.0,0.0,0.3959999978542328,0.4152640104293823,0.40332479774951924,0.40332479774951924,0.006942401431914052,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014175999909639359,0.03379200026392937,0.020595200080424547,0.019600000232458115,0.006548754248881344,0.026192623739694512,0.040006882507168426,0.029155180178492036,0.029155180178492036,0.00408828748438241,0.024591375524545756,0.037561119537986146,0.027372820355088746,0.027372820355088746,0.0038383559348575944,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,72,,,,,,,,False,,,64,BF16,generic,none,CUDA_EVENT,True,[64],[0],"[512, 512, 512, 512, 512, 512, 512, 512]",1,8,8,9,8,512.0,0.1111111111111111,q64_8q1s512,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.013856000266969204,0.020896000787615776,0.015318400040268899,0.014431999996304512,0.0019640312960926966,0.029349018208693862,0.06236262941356679,0.03714313592014963,0.03714313592014963,0.009618313791872569,0.028826981462526914,0.06125337308649041,0.036482463672774496,0.036482463672774496,0.009447230957011863,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,136,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,8,8,9,8,1024.0,0.1111111111111111,q128_8q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.013856000266969204,0.032127998769283295,0.017286399938166143,0.014479999896138906,0.005536495446636193,0.03273085874558354,0.04394578491937082,0.03563837490653034,0.03563837490653034,0.003429601730256567,0.03488514202593899,0.04683821345079978,0.037984025407085474,0.037984025407085474,0.0036553316361902684,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,144,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,16,16,17,16,1024.0,0.058823529411764705,q128_16q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.013856000266969204,0.021503999829292297,0.015078400075435639,0.01425600005313754,0.002238435975478849,0.042100813549974185,0.051784144690147756,0.045378692890289625,0.045378692890289625,0.003184109940650555,0.05111518843748309,0.06287185663927425,0.05509490773570219,0.05509490773570219,0.0038658725544299132,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,272,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,16,16,17,16,2048.0,0.058823529411764705,q256_16q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.02687999978661537,0.017667199857532977,0.01566399959847331,0.003864811846387173,0.048475323773821306,0.05599956978723733,0.05123499252968345,0.05123499252968345,0.002427379820423941,0.08429268135885387,0.09737642835214408,0.0890914090616175,0.0890914090616175,0.0042209177332077005,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,288,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,32,32,33,32,2048.0,0.030303030303030304,q256_32q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01462399959564209,0.03359999880194664,0.019804799742996693,0.016032000072300434,0.006750519908145474,0.07121508474579985,0.07292308367136396,0.07194410845270183,0.07194410845270183,0.0005500065915943844,0.14760091249712767,0.15114092373010238,0.14911189243564577,0.14911189243564577,0.0011399477384397005,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,544,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q512_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.021856000646948814,0.016835200227797033,0.015263999812304974,0.00283854448975065,0.08146338272142935,0.08560865714384096,0.0834932625520734,0.0834932625520734,0.0012259635905347874,0.2782486219401307,0.29240733788179385,0.2851819365989658,0.2851819365989658,0.004187435731481665,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,576,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,64,64,65,64,4096.0,0.015384615384615385,q512_64q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.04495999962091446,0.02143679987639189,0.015856000129133463,0.009709987872683342,0.1194459208702178,0.12302524755181463,0.12053885718696096,0.12053885718696096,0.001005118119341339,0.5597220650459199,0.5764947443228802,0.5648435507167102,0.5648435507167102,0.004709970715400788,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192]",1,64,64,65,64,8192.0,0.015384615384615385,q1k_64q1s8k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018112000077962875,0.026496000587940216,0.019318400137126445,0.018432000651955605,0.0024249605112359905,0.19770253574610402,0.222811797868348,0.2054811520619412,0.2054811520619412,0.008000944254868043,0.13941746080159492,0.157124211777114,0.14490284788179197,0.14490284788179197,0.005642170080515992,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q2k_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.02595200017094612,0.03155200183391571,0.02727359998971224,0.026575999334454536,0.0016260948764843166,0.6014684881116659,0.6095472988212,0.6046757700946376,0.6046757700946376,0.0023537435563177303,0.11325152264578045,0.11477269562841545,0.11385542721485654,0.11385542721485654,0.0004431903698023628,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,16,16,17,16,4096.0,0.058823529411764705,q4k_16q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014303999952971935,0.03359999880194664,0.01814719969406724,0.015375999733805656,0.005678506740662529,0.06774400174617767,0.07891199737787247,0.07312640026211739,0.07312640026211739,0.003826583051422731,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,2,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512]",1024,512,512,512.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01788800023496151,0.028543999418616295,0.021340799890458582,0.019504000432789326,0.003718627037219618,0.09548799693584442,0.1327359974384308,0.10823359936475753,0.10823359936475753,0.013230635292043864,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,4,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512]",2048,512,512,512.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02627200074493885,0.030368000268936157,0.027184000052511693,0.02643200010061264,0.0013545679205210022,0.14364799857139587,0.1597760021686554,0.15008639842271806,0.15008639842271806,0.005367723415171222,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512]",4096,512,512,512.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04150399938225746,0.04726399853825569,0.042950399965047834,0.04224000126123428,0.001720147501765378,0.2433920055627823,0.28963199257850647,0.2549152016639709,0.2549152016639709,0.013450991019687407,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512]",8192,512,512,512.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.025887999683618546,0.03407999873161316,0.028303999826312064,0.026367999613285065,0.0028877495368841042,0.325439989566803,0.3441599905490875,0.33442879617214205,0.33442879617214205,0.005693597811441922,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,2,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048]",4096,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04137599840760231,0.05084799975156784,0.044828799366950986,0.043087998405098915,0.003560363865797613,0.6110399961471558,0.6421759724617004,0.6204223990440368,0.6204223990440368,0.011214490370794758,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,4,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048, 2048, 2048]",8192,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014816000126302242,0.02364799939095974,0.016668799985200166,0.01592000015079975,0.0025515238179941247,0.0,0.0,0.0,0.0,0.0,0.0544000007212162,0.09014400094747543,0.06511679962277411,0.06511679962277411,0.013005726711295521,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.0208320003002882,0.016604799963533878,0.01566399959847331,0.0021353594266203244,0.0,0.0,0.0,0.0,0.0,0.17871999740600586,0.1961279958486557,0.18568639904260634,0.18568639904260634,0.004566142885274747,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014431999996304512,0.02006400004029274,0.015619200002402068,0.01508800033479929,0.001572984783715635,0.0,0.0,0.0,0.0,0.0,0.2730880081653595,0.29721599817276,0.2815328001976013,0.2815328001976013,0.007016534727616923,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.02755199931561947,0.018678399827331306,0.015919999685138464,0.004323555372382858,0.0,0.0,0.0,0.0,0.0,0.3893119990825653,0.44041600823402405,0.40225600004196166,0.40225600004196166,0.013442998165884852,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014303999952971935,0.02659199945628643,0.016748800035566093,0.015343999955803156,0.0036412341868394082,0.0,0.0,0.0,0.0,0.0,0.7404800057411194,0.9689919948577881,0.8451807916164398,0.8451807916164398,0.08637455920036795,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014751999638974667,0.02160000056028366,0.016752000153064727,0.01521599991247058,0.0025167650350367246,0.0,0.0,0.0,0.0,0.0,0.06412799656391144,0.08956799656152725,0.07242240011692047,0.07242240011692047,0.009797120588901621,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.022911999374628067,0.016137600038200618,0.015343999955803156,0.002330911555470513,0.0,0.0,0.0,0.0,0.0,0.3128319978713989,0.3282879889011383,0.3203647971153259,0.3203647971153259,0.004598568238907996,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.03574400022625923,0.01919359974563122,0.01583999954164028,0.006642521913489215,0.0,0.0,0.0,0.0,0.0,0.5050879716873169,0.5311999917030334,0.511932796239853,0.511932796239853,0.007150929647317677,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.02595200017094612,0.016992000117897987,0.015456000342965126,0.003390731937723684,0.0,0.0,0.0,0.0,0.0,0.7385600209236145,0.7681919932365417,0.7483008027076722,0.7483008027076722,0.010214373356208012,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.038975998759269714,0.01952639976516366,0.015327999833971262,0.007695927285621061,0.0,0.0,0.0,0.0,0.0,1.4228800535202026,1.5237760543823242,1.4435008168220522,1.4435008168220522,0.0342955291725742,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.03907199949026108,0.01793599994853139,0.015248000156134367,0.007167103363234667,0.0,0.0,0.0,0.0,0.0,0.06947200000286102,0.08857599645853043,0.07469440028071403,0.07469440028071403,0.00583249952929747,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.02175999991595745,0.016710399929434062,0.015327999833971262,0.002705383977079183,0.0,0.0,0.0,0.0,0.0,0.388480007648468,0.4073280096054077,0.3966591984033584,0.3966591984033584,0.004487315516095964,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,8,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.0144640002399683,0.022048000246286392,0.017011200170964004,0.015392000321298838,0.002776899649845722,0.0,0.0,0.0,0.0,0.0,0.622048020362854,0.6347839832305908,0.6262047946453094,0.6262047946453094,0.004593360122958751,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,16,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014592000283300877,0.02937600016593933,0.01847040019929409,0.015344000421464443,0.005043809142122341,0.0,0.0,0.0,0.0,0.0,0.9105280041694641,0.9304640293121338,0.914108806848526,0.914108806848526,0.00563919268816644,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,32,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01462399959564209,0.039903998374938965,0.01791359977796674,0.015440000221133232,0.007367547649297217,0.0,0.0,0.0,0.0,0.0,1.764799952507019,1.7965760231018066,1.770739197731018,1.770739197731018,0.009082122770320404,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,64,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015168000012636185,0.03187200054526329,0.018553599901497363,0.016127999871969223,0.0050628774268006264,0.16963527081512533,0.1737871320906266,0.17088926838108623,0.17088926838108623,0.0011703114258121128,0.47362872483230506,0.48522089403242025,0.47712993814280913,0.47712993814280913,0.0032675581298664976,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,520,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,8,8,9,8,16384.0,0.1111111111111111,q512_8q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.03868800029158592,0.01921919994056225,0.016959999687969685,0.006691309533030663,0.1589600576212269,0.16117782913137854,0.15969057520605917,0.15969057520605917,0.0006212984448748182,0.5199519263456005,0.5272061673552852,0.5223414198520004,0.5223414198520004,0.002032242112153396,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1040,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,16,16,17,16,16384.0,0.058823529411764705,q1k_16q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018400000408291817,0.024320000782608986,0.019910399988293647,0.0191040001809597,0.001841532763984352,0.25650752966102763,0.27066608538463055,0.259736894547936,0.259736894547936,0.00462927315947332,0.5278764825612624,0.5570139063374764,0.5345223138928448,0.5345223138928448,0.009526755161797178,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,32,32,33,32,16384.0,0.030303030303030304,q2k_32q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.026048000901937485,0.03728000074625015,0.028636799938976765,0.02705600019544363,0.003313953649011259,0.9270006318443208,0.9712965120641314,0.93480767601568,0.93480767601568,0.012612551450284608,0.8181833128578277,0.8572794566782395,0.8250739157811953,0.8250739157811953,0.01113200873299586,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,16,16,17,16,32768.0,0.058823529411764705,q4k_16q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.018079999834299088,0.037696000188589096,0.022092800214886667,0.019024000503122807,0.005986278786633071,0.2839857165542317,0.2876110048757805,0.2849506939696605,0.2849506939696605,0.0009995178425916847,1.0871823008331585,1.101060989333509,1.0908765231323903,1.0908765231323903,0.0038264533900426207,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,32,32,33,32,32768.0,0.030303030303030304,q2k_32q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.015072000212967396,0.025567999109625816,0.01726400014013052,0.015728000551462173,0.0031324465940990903,0.137270464802061,0.13799613818579368,0.13751499486424038,0.13751499486424038,0.00025221761530472904,2.3021855212208884,2.314355908780759,2.3062865750744197,2.3062865750744197,0.004229983070201486,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,64,64,65,64,32768.0,0.015384615384615385,q1k_64q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014560000039637089,0.022784000262618065,0.016435200069099664,0.015519999898970127,0.0023688301421469523,0.04879999905824661,0.09139200299978256,0.06054079942405224,0.06054079942405224,0.012152608702448775,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,64,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,False,0.0,64.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014816000126302242,0.02844800055027008,0.017008000146597625,0.015696000307798386,0.003941466294662679,0.047807998955249786,0.07356800138950348,0.05626560002565384,0.05626560002565384,0.00842179125412236,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,128,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,False,0.0,128.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.053408000618219376,0.019353600218892097,0.01532800029963255,0.01138302400841626,0.048448000103235245,0.0785600021481514,0.0556256003677845,0.0556256003677845,0.009348274502616908,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,256,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,False,0.0,256.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015039999969303608,0.03139200061559677,0.01823679991066456,0.016159999649971724,0.004860071238302512,0.055424001067876816,0.08505599945783615,0.0640383992344141,0.0640383992344141,0.00921448636178623,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,512,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,False,0.0,512.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014751999638974667,0.026528000831604004,0.017324799951165915,0.01536000007763505,0.0036004822686428305,0.07660800218582153,0.0942080020904541,0.08209280073642732,0.08209280073642732,0.00515895587669752,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.021247999742627144,0.016403199825435876,0.01532800029963255,0.001995897022647934,0.11395200341939926,0.15113599598407745,0.12431039959192276,0.12431039959192276,0.011164431123683732,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.017184000462293625,0.028672000393271446,0.019865600019693376,0.017823999747633934,0.0035798491315929977,0.3171840012073517,0.3341119885444641,0.3261695951223373,0.3261695951223373,0.005111046452878918,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024639999493956566,0.030400000512599945,0.02656640000641346,0.02556800004094839,0.0020189846603237303,1.0648640394210815,1.0828479528427124,1.0715327858924866,1.0715327858924866,0.005558639263049851,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014976000413298607,0.021695999428629875,0.017305599898099898,0.01593599934130907,0.0025718046517268054,0.04956800118088722,0.07932800054550171,0.06228480041027069,0.06228480041027069,0.01027160349757827,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,64,448.0,True,FLASH_ATTN,False,vllm020_batch_spec,[64],64,64,64,64.0,True,0.0,0.0,0.0,True,448.0,512.0,64,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q64s512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01539199985563755,0.03580800071358681,0.019686400331556796,0.017136000096797943,0.005918387811304003,0.05158400163054466,0.10713600367307663,0.06364160068333148,0.06364160068333148,0.015832957809696766,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,128,896.0,True,FLASH_ATTN,False,vllm020_batch_spec,[128],128,128,128,128.0,True,0.0,0.0,0.0,True,896.0,1024.0,128,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q128s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015168000012636185,0.02223999984562397,0.016672000009566545,0.015887999907135963,0.0020934945946034563,0.06521599739789963,0.08902399986982346,0.07520959973335266,0.07520959973335266,0.007913840684102929,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,256,1792.0,True,FLASH_ATTN,False,vllm020_batch_spec,[256],256,256,256,256.0,True,0.0,0.0,0.0,True,1792.0,2048.0,256,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q256s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015231999568641186,0.04028800129890442,0.019971200078725816,0.01646399963647127,0.007351305886577286,0.14467200636863708,0.16844800114631653,0.15363519936800005,0.15363519936800005,0.008174435899956223,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,512,3584.0,True,FLASH_ATTN,False,vllm020_batch_spec,[512],512,512,512,512.0,True,0.0,0.0,0.0,True,3584.0,4096.0,512,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q512s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015519999898970127,0.02304000034928322,0.01775679988786578,0.016784000210464,0.0025077243712082584,0.33740800619125366,0.35343998670578003,0.3445120006799698,0.3445120006799698,0.00445648463590358,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,1024,7168.0,True,FLASH_ATTN,False,vllm020_batch_spec,[1024],1024,1024,1024,1024.0,True,0.0,0.0,0.0,True,7168.0,8192.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1ks8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015647999942302704,0.02393599972128868,0.01809599995613098,0.01654400024563074,0.002998393102466254,0.0,0.0,0.0,0.0,0.0,0.0504320003092289,0.0843840017914772,0.060083200410008426,0.060083200410008426,0.00986959318572296,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01603199914097786,0.030047999694943428,0.019510399922728537,0.017680000513792038,0.004065185265963332,0.0,0.0,0.0,0.0,0.0,0.05004800111055374,0.07036799937486649,0.059315200522542,0.059315200522542,0.006537768821329647,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.02393599972128868,0.018035199772566558,0.016512000001966953,0.0030667689116777724,0.0,0.0,0.0,0.0,0.0,0.05100800096988678,0.06735999882221222,0.058387200161814694,0.058387200161814694,0.005832787739241381,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01500799972563982,0.026208000257611275,0.017500799987465142,0.01646399963647127,0.0030666103306165714,0.0,0.0,0.0,0.0,0.0,0.05023999884724617,0.07254400104284286,0.057254400476813315,0.057254400476813315,0.006890853653068151,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.027327999472618103,0.0179776000790298,0.01648000068962574,0.003783417447531392,0.0,0.0,0.0,0.0,0.0,0.04819199815392494,0.07100799679756165,0.05621119923889638,0.05621119923889638,0.007824391484124725,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,128.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s128,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015647999942302704,0.036448001861572266,0.019136000238358975,0.016704000532627106,0.006076530095624803,0.0,0.0,0.0,0.0,0.0,0.047488000243902206,0.08441600203514099,0.0588383998721838,0.0588383998721838,0.011275940726332522,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015135999768972397,0.033952001482248306,0.02119360016658902,0.018000000156462193,0.007136646614305304,0.0,0.0,0.0,0.0,0.0,0.04931199923157692,0.06992000341415405,0.05755840018391609,0.05755840018391609,0.007297587694315226,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.034272000193595886,0.0204415999352932,0.017311999574303627,0.00695404701803013,0.0,0.0,0.0,0.0,0.0,0.05215999856591225,0.0735040009021759,0.06117440015077591,0.06117440015077591,0.007381118436894922,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015359999611973763,0.03743999823927879,0.02012479966506362,0.01775999926030636,0.006181045509079057,0.0,0.0,0.0,0.0,0.0,0.06355199962854385,0.08508799970149994,0.07019200026988984,0.07019200026988984,0.0062246580442117845,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,1024.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s1k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015552000142633915,0.032607998698949814,0.01959999995306134,0.017487999983131886,0.004882475068460749,0.0,0.0,0.0,0.0,0.0,0.04918399825692177,0.0865280032157898,0.05973760038614274,0.05973760038614274,0.011546847942113974,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015168000012636185,0.02675200067460537,0.017430400010198355,0.016560000367462635,0.003243135094239819,0.0,0.0,0.0,0.0,0.0,0.052671998739242554,0.08367999643087387,0.06238719932734965,0.06238719932734965,0.011207824780630104,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01500799972563982,0.03612799942493439,0.019327999837696553,0.01688000001013279,0.006058251425153085,0.0,0.0,0.0,0.0,0.0,0.06364800035953522,0.07878399640321732,0.06935679838061332,0.06935679838061332,0.004515588328677643,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015104000456631184,0.03711999952793121,0.019670399930328132,0.017152000218629837,0.006165994646522264,0.0,0.0,0.0,0.0,0.0,0.09071999788284302,0.11507199704647064,0.10252480059862136,0.10252480059862136,0.008782544051535657,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,2048.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015039999969303608,0.026528000831604004,0.018396800104528665,0.01601599995046854,0.004279788048986283,0.0,0.0,0.0,0.0,0.0,0.05331199988722801,0.07977599650621414,0.06076480001211167,0.06076480001211167,0.008761579122069606,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015359999611973763,0.02643200010061264,0.01726400004699826,0.015855999663472176,0.003210008417242029,0.0,0.0,0.0,0.0,0.0,0.062431998550891876,0.08246400207281113,0.06970879957079888,0.06970879957079888,0.007016341676068233,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014911999925971031,0.02223999984562397,0.0166015999391675,0.015887999907135963,0.00204913969129354,0.0,0.0,0.0,0.0,0.0,0.10127999633550644,0.1141119971871376,0.10621120035648347,0.10621120035648347,0.004029318311754605,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015359999611973763,0.03363199904561043,0.019305599946528675,0.016671999357640743,0.0055445582307981234,0.0,0.0,0.0,0.0,0.0,0.1319040060043335,0.15839999914169312,0.14040640145540234,0.14040640145540234,0.007998041073596942,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,4096.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s4k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014368000440299511,0.04447999969124794,0.018908800091594458,0.0157279996201396,0.0086814937461721,0.0,0.0,0.0,0.0,0.0,0.06428799778223038,0.08982399851083755,0.07349760085344315,0.07349760085344315,0.008605284512197258,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015104000456631184,0.028672000393271446,0.01890560006722808,0.016080000437796116,0.004797584627815021,0.0,0.0,0.0,0.0,0.0,0.11123199760913849,0.14115199446678162,0.11942399889230729,0.11942399889230729,0.008513394706791027,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014271999709308147,0.024927999824285507,0.016915200091898442,0.015216000378131866,0.003374147499442635,0.0,0.0,0.0,0.0,0.0,0.15887999534606934,0.18111999332904816,0.16934399753808976,0.16934399753808976,0.007415181260761123,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014336000196635723,0.026944000273942947,0.017737600207328796,0.015199999790638685,0.00415598714375255,0.0,0.0,0.0,0.0,0.0,0.2192319929599762,0.23472000658512115,0.22809920012950893,0.22809920012950893,0.004730841376327335,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,8192.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s8k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.05052800104022026,0.020320000313222408,0.015520000364631414,0.010604522177924678,0.024270629882498958,0.03340247625954076,0.028446344104128624,0.028446344104128624,0.0032330369099793834,0.023697370291069768,0.03261352722995356,0.027774456325453972,0.027774456325453972,0.0031566742680923386,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,72,,,,,,,,False,,,64,BF16,generic,none,CUDA_EVENT,True,[64],[0],"[512, 512, 512, 512, 512, 512, 512, 512]",1,8,8,9,8,512.0,0.1111111111111111,q64_8q1s512,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.015359999611973763,0.033663999289274216,0.019987199828028678,0.018240000121295452,0.005522929139253732,0.0259194055660947,0.03719755183990719,0.029916029687899703,0.029916029687899703,0.003966666590554318,0.027104595853449518,0.03889844644729374,0.03128396954022015,0.03128396954022015,0.004148046317967881,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,136,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,8,8,9,8,1024.0,0.1111111111111111,q128_8q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014592000283300877,0.03299200162291527,0.019840000104159115,0.016207999549806118,0.006546343008726814,0.02587869595769926,0.03763167265431482,0.030174939058162802,0.030174939058162802,0.0037303581782277364,0.026473304070195713,0.03849632587654989,0.03086826083864964,0.03086826083864964,0.003816069654529222,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,144,,,,,,,,False,,,128,BF16,generic,none,CUDA_EVENT,True,[128],[0],"[1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024]",1,16,16,17,16,1024.0,0.058823529411764705,q128_16q1s1k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.013824000023305416,0.021023999899625778,0.016304000187665223,0.015584000386297703,0.0023236165806545476,0.031810621525966996,0.04156949936878106,0.0346749350032807,0.0346749350032807,0.003130209181143985,0.035677378270900374,0.04662250161636451,0.038889864871740294,0.038889864871740294,0.0035107033960828727,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,272,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,16,16,17,16,2048.0,0.058823529411764705,q256_16q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.02175999991595745,0.01569600012153387,0.015008000191301107,0.0020882666168036863,0.038211712107062853,0.07212229256520057,0.04364509673334097,0.04364509673334097,0.009671953074025085,0.0476442859917874,0.08992570455184198,0.05441890342616107,0.05441890342616107,0.01205947791784038,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,288,,,,,,,,False,,,256,BF16,generic,none,CUDA_EVENT,True,[256],[0],"[2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048]",1,32,32,33,32,2048.0,0.030303030303030304,q256_32q1s2k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014368000440299511,0.033824000507593155,0.017628800217062236,0.014864000026136637,0.005791495403857738,0.05214261250030033,0.06266261508706669,0.05677791295527661,0.05677791295527661,0.0030298223079651514,0.08648138506878382,0.10392938682791132,0.09416928531647478,0.09416928531647478,0.005025126612204489,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,544,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q512_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.031199999153614044,0.017900799959897996,0.015343999955803156,0.004974475661316249,0.06411958891421111,0.07405276123263956,0.06793047918211377,0.06793047918211377,0.0033918658556700006,0.14058441263169497,0.16236323591492058,0.1489399211274489,0.1489399211274489,0.00743678300375353,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,576,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,64,64,65,64,4096.0,0.015384615384615385,q512_64q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.03855999931693077,0.018495999928563833,0.015647999942302704,0.006918530056761627,0.09872985549401277,0.10683454583043753,0.10185316839884552,0.10185316839884552,0.0020802254037042074,0.2743261390166379,0.2968454508984596,0.2830044295482752,0.2830044295482752,0.005780016596065092,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192, 8192]",1,64,64,65,64,8192.0,0.015384615384615385,q1k_64q1s8k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014911999925971031,0.030400000512599945,0.01977920001372695,0.01726400014013052,0.005350883464206169,0.13918872472233365,0.16279522855335207,0.1501912864839173,0.1501912864839173,0.008992185529011513,0.11892328861766266,0.13909276049083735,0.1283239123428725,0.1283239123428725,0.007682951884956939,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,32,32,33,32,4096.0,0.030303030303030304,q2k_32q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01727999933063984,0.02223999984562397,0.01819519978016615,0.017680000513792038,0.0014397298270620873,0.3728044181625443,0.38295504353701676,0.3761264483787333,0.3761264483787333,0.0033660272332490977,0.07967557017401883,0.08184495665371805,0.08038555277807365,0.08038555277807365,0.0007193856241088455,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096, 4096]",1,16,16,17,16,4096.0,0.058823529411764705,q4k_16q1s4k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01500799972563982,0.034591998904943466,0.01941439984366298,0.01756799966096878,0.005541380584373771,0.05951999872922897,0.08268799632787704,0.06715519949793816,0.06715519949793816,0.006657802116288364,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,2,1024,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512]",1024,512,512,512.0,True,0.0,0.0,0.0,False,0.0,1024.0,1024,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015104000456631184,0.03283200040459633,0.019180799927562477,0.016543999314308167,0.0052187911233635975,0.07199999690055847,0.08675199747085571,0.07749439924955369,0.07749439924955369,0.004849874741156772,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,4,2048,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512]",2048,512,512,512.0,True,0.0,0.0,0.0,False,0.0,2048.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01724799908697605,0.02284800074994564,0.01928640007972717,0.018400000408291817,0.0020641390157565697,0.09676799923181534,0.12310399860143663,0.10618879944086074,0.10618879944086074,0.008982738207839057,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512]",4096,512,512,512.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024224000051617622,0.03315199911594391,0.027436799928545953,0.027328000403940678,0.002679154874546328,0.14716799557209015,0.16412800550460815,0.15470399856567385,0.15470399856567385,0.005423454433987419,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512]",8192,512,512,512.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q512,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01833599992096424,0.04806400090456009,0.026252799853682517,0.022672000341117382,0.00878438440429033,0.1844799965620041,0.2072959989309311,0.19359359890222552,0.19359359890222552,0.007663539820967659,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,2,4096,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048]",4096,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,4096.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,2q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024480000138282776,0.04979199916124344,0.029081599973142146,0.025679999962449074,0.007227422044689197,0.34147199988365173,0.37968000769615173,0.3583200007677078,0.3583200007677078,0.011804422100726231,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,4,8192,0.0,True,FLASH_ATTN,False,vllm020_batch_spec,"[2048, 2048, 2048, 2048]",8192,2048,2048,2048.0,True,0.0,0.0,0.0,False,0.0,8192.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,4q2k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014720000326633453,0.023744000121951103,0.017753600236028434,0.016368000768125057,0.003061040281616705,0.0,0.0,0.0,0.0,0.0,0.05686400085687637,0.1090880036354065,0.06629760004580021,0.06629760004580021,0.014803727026812366,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.021088000386953354,0.01647359998896718,0.015840000472962856,0.0017964402433206,0.0,0.0,0.0,0.0,0.0,0.08540800213813782,0.09961599856615067,0.09146559983491898,0.09146559983491898,0.00447423706371901,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014560000039637089,0.04156799986958504,0.018512000143527985,0.015584000386297703,0.007849701681877904,0.0,0.0,0.0,0.0,0.0,0.17948800325393677,0.194815993309021,0.1866239994764328,0.1866239994764328,0.004156328241374654,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.029632000252604485,0.019449600111693145,0.016736000776290894,0.005126811425136928,0.0,0.0,0.0,0.0,0.0,0.27529600262641907,0.2898879945278168,0.2825664013624191,0.2825664013624191,0.004542801609674093,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014399999752640724,0.031328000128269196,0.019759999960660933,0.016367999836802483,0.006266303740589241,0.0,0.0,0.0,0.0,0.0,0.39190399646759033,0.4079039990901947,0.3987520009279252,0.3987520009279252,0.0037367727509362725,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,16384.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01539199985563755,0.040031999349594116,0.0212032001465559,0.01649599988013506,0.008068547381446682,0.0,0.0,0.0,0.0,0.0,0.06588800251483917,0.07897599786520004,0.0703904002904892,0.0703904002904892,0.004214826869769361,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014495999552309513,0.022495999932289124,0.016006399970501663,0.01515199989080429,0.002265581884342885,0.0,0.0,0.0,0.0,0.0,0.1231679990887642,0.13526399433612823,0.1288223996758461,0.1288223996758461,0.004328976434897094,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014592000283300877,0.023231999948620796,0.01705600004643202,0.016048000194132328,0.002810904312746391,0.0,0.0,0.0,0.0,0.0,0.3158079981803894,0.33129599690437317,0.32348800003528594,0.32348800003528594,0.00417599076136664,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014527999795973301,0.021888000890612602,0.01708160014823079,0.01609600055962801,0.0025289234621475062,0.0,0.0,0.0,0.0,0.0,0.5103679895401001,0.5200319886207581,0.5132320046424866,0.5132320046424866,0.0035759433157749533,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014688000082969666,0.032416000962257385,0.019353600032627583,0.016176000237464905,0.0061458798960684425,0.0,0.0,0.0,0.0,0.0,0.7400320172309875,0.7516480088233948,0.7449311971664427,0.7449311971664427,0.004573461776168607,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,32768.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.0144640002399683,0.02006400004029274,0.01573119992390275,0.014992000069469213,0.001654446341262991,0.0,0.0,0.0,0.0,0.0,0.07097599655389786,0.09932799637317657,0.07828159928321837,0.07828159928321837,0.009464602895232642,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,[1],1,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.02319999970495701,0.016883199848234654,0.015775999519973993,0.0026693906156048403,0.0,0.0,0.0,0.0,0.0,0.14176000654697418,0.1598079949617386,0.15063679963350293,0.15063679963350293,0.005741662969679433,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,8,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1]",8,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,8q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01484800036996603,0.03203200176358223,0.022224000189453363,0.02247999981045723,0.005996176183124981,0.0,0.0,0.0,0.0,0.0,0.39180800318717957,0.41046398878097534,0.4003200054168701,0.4003200054168701,0.005552433764307601,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,16,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",16,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,16q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015072000212967396,0.028192000463604927,0.017152000125497578,0.015536000020802021,0.003847486121602065,0.0,0.0,0.0,0.0,0.0,0.6239359974861145,0.6367359757423401,0.6285343945026398,0.6285343945026398,0.004006084286606002,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,32,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",32,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,32q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014879999682307243,0.026688000187277794,0.01718079997226596,0.015488000120967627,0.0037132536813398722,0.0,0.0,0.0,0.0,0.0,0.9141119718551636,0.9721279740333557,0.927455997467041,0.927455997467041,0.02148767779232235,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,64,0,40960.0,False,FLASH_ATTN,False,vllm020_batch_spec,"[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]",64,1,1,1.0,True,0.0,0.0,0.0,False,0,0,0,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,64q1s40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014751999638974667,0.047520000487565994,0.028710400220006704,0.0266720000654459,0.010774688401598903,0.06094816381288764,0.06816969726785131,0.0644060660218149,0.0644060660218149,0.002203106589073183,0.087051838094461,0.09736630405680231,0.09199073574791852,0.09199073574791852,0.0031466818046499644,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,9,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,520,,,,,,,,False,,,512,BF16,generic,none,CUDA_EVENT,True,[512],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,8,8,9,8,16384.0,0.1111111111111111,q512_8q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.014911999925971031,0.02489599958062172,0.017427200078964235,0.01595200039446354,0.003071906489592143,0.19733789497223914,0.2016295616652644,0.19860388994013833,0.19860388994013833,0.001512389485439305,0.44861409134062713,0.45837046456077934,0.45149211526120153,0.45149211526120153,0.0034381598874302305,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1040,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,16,16,17,16,16384.0,0.058823529411764705,q1k_16q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.015039999969303608,0.03046399913728237,0.020643199887126686,0.016944000497460365,0.006320148675595729,0.2157924314537054,0.22215709640166995,0.21769400765743155,0.21769400765743155,0.0019679921844645526,0.4905115822753901,0.5049789194903589,0.4948340005651485,0.4948340005651485,0.0044733865493072344,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384, 16384]",1,32,32,33,32,16384.0,0.030303030303030304,q2k_32q1s16k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.017152000218629837,0.02425600029528141,0.01865920014679432,0.018112000077962875,0.0019454553198986453,0.7430866512973927,0.7578834738720781,0.7455351005236347,0.7455351005236347,0.00416692487556653,0.7369773831645824,0.7516525540362471,0.7394057025273603,0.7394057025273603,0.004132666607961175,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,17,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,4112,,,,,,,,False,,,4096,BF16,generic,none,CUDA_EVENT,True,[4096],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,16,16,17,16,32768.0,0.058823529411764705,q4k_16q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.015263999812304974,0.02470399998128414,0.018009600043296815,0.01657600048929453,0.0031544532774534346,0.25251173919752334,0.25566890792461106,0.2536729054481981,0.2536729054481981,0.0010279562245342983,1.0425282722084215,1.0555630628624373,1.0473223013846877,1.0473223013846877,0.004244053880724073,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,33,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,2080,,,,,,,,False,,,2048,BF16,generic,none,CUDA_EVENT,True,[2048],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,32,32,33,32,32768.0,0.030303030303030304,q2k_32q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.01500799972563982,0.027807999402284622,0.01912960009649396,0.01643200032413006,0.004688164695934997,0.1251379565220268,0.14341821167748953,0.12742497577885914,0.12742497577885914,0.005353812167343766,1.1355340167064276,1.3014137556763312,1.1562870179154463,1.1562870179154463,0.048581869194943325,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,65,0,0,True,FLASH_ATTN,False,true_mixed_fused_projected,,1088,,,,,,,,False,,,1024,BF16,generic,none,CUDA_EVENT,True,[1024],[0],"[32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768, 32768]",1,64,64,65,64,32768.0,0.015384615384615385,q1k_64q1s32k,fused_total_conserving_projection_by_same_tp_pure_prefill_decode_reference_ratio
+0.0,0.0,0.0,0.0,0.0,0.07446400076150894,0.08246400207281113,0.07736000046133995,0.07713599875569344,0.002301031358835964,11.935359954833984,12.102368354797363,11.96896333694458,11.96896333694458,0.05162549713370252,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,8192,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,8192.0,16384.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07664000242948532,0.08118399977684021,0.07796800062060356,0.077504001557827,0.0013681081693640953,19.84774398803711,20.15795135498047,19.915702438354494,19.915702438354494,0.0912299324030902,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,8192,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,16384.0,24576.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks24k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.0753600001335144,0.08207999914884567,0.07715519964694977,0.07595199719071388,0.0024238358447475007,27.781503677368164,27.9836483001709,27.844886589050287,27.844886589050287,0.061307010154819624,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,8192,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,24576.0,32768.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04339199885725975,0.050016000866889954,0.04504639990627766,0.04391999915242195,0.002339637813151782,5.1544318199157715,5.269279956817627,5.16938238143921,5.16938238143921,0.03363105774712194,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,4096,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,8192.0,12288.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks12k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04291199892759323,0.058848001062870026,0.045657599717378615,0.04383999854326248,0.004522763233004815,9.256383895874023,9.27734375,9.261776161193849,9.261776161193849,0.0062232312957398745,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,4096,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,16384.0,20480.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks20k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.043296001851558685,0.04982399940490723,0.045123199746012685,0.04387199878692627,0.0023369398894319345,13.365216255187988,13.634464263916016,13.398719978332519,13.398719978332519,0.0787651922310212,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,4096,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,24576.0,28672.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks28k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02703999914228916,0.0297279991209507,0.027692800015211107,0.02723200060427189,0.000851127720159845,2.387968063354492,2.4014720916748047,2.3920736074447637,2.3920736074447637,0.003552645719470337,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,2048,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,8192.0,10240.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks10k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02691200003027916,0.029823999851942062,0.027689599990844728,0.027583999559283257,0.0007783369959809023,4.440767765045166,4.4521918296813965,4.444268751144409,4.444268751144409,0.004022325406840951,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,2048,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,16384.0,18432.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks18k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02703999914228916,0.03888000175356865,0.030262400023639204,0.02801600005477667,0.0037877256209144718,6.500351905822754,6.600607872009277,6.524902391433716,6.524902391433716,0.03321667087212851,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,2048,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,24576.0,26624.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks26k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.07865600287914276,0.08668799698352814,0.08130879923701287,0.07993599772453308,0.0027016201268628523,35.717376708984375,35.843265533447266,35.771837615966795,35.771837615966795,0.04024815379149301,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,1,40960,1,8192,32768.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,32768.0,40960.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04179200157523155,0.04864000156521797,0.0436256006360054,0.04267200082540512,0.002191476789190783,6.103871822357178,6.18287992477417,6.118390369415283,6.118390369415283,0.022924175047267112,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,8192,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,8192.0,16384.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04163200035691261,0.05987200140953064,0.04625920057296753,0.04403200000524521,0.005544136325859629,10.207136154174805,11.073247909545898,10.326777648925782,10.326777648925782,0.25139335734700863,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,8192,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,16384.0,24576.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks24k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04137599840760231,0.05027199909090996,0.04355199970304966,0.042399998754262924,0.0026572716942034787,14.303423881530762,14.343520164489746,14.311049747467042,14.311049747467042,0.01137511287661936,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,8192,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,24576.0,32768.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.026016000658273697,0.03718400001525879,0.03212799951434135,0.03270399942994118,0.003953184738275612,2.6534719467163086,2.6875839233398438,2.661257576942444,2.661257576942444,0.009372641904054613,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,4096,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,8192.0,12288.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks12k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.025887999683618546,0.03347200155258179,0.027168000116944313,0.02649599965661764,0.0021492000999850562,4.70630407333374,4.7400641441345215,4.714492845535279,4.714492845535279,0.009349196197962609,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,4096,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,16384.0,20480.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks20k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.025631999596953392,0.030719999223947525,0.02736639976501465,0.02711999975144863,0.0013921874495504173,6.759712219238281,6.831999778747559,6.774675178527832,6.774675178527832,0.021507023842020512,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,4096,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,24576.0,28672.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks28k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.017855999991297722,0.024831999093294144,0.01991359982639551,0.018655999563634396,0.0023971416189370203,1.3446400165557861,1.3609600067138672,1.3508928060531615,1.3508928060531615,0.006186954712541735,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,2048,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,8192.0,10240.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks10k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.0180479995906353,0.022752000018954277,0.019507200084626676,0.01896000001579523,0.0015079573553847933,2.5208001136779785,2.5887041091918945,2.534086418151855,2.534086418151855,0.01891953046799999,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,2048,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,16384.0,18432.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks18k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.018144000321626663,0.03222399950027466,0.02052800003439188,0.018864000216126442,0.004114364828004189,3.6922879219055176,3.760576009750366,3.706630396842957,3.706630396842957,0.019888657921309623,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,2048,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,24576.0,26624.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks26k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.04169600084424019,0.04822399839758873,0.0439775999635458,0.04334400035440922,0.002101844179466219,18.41494369506836,18.502975463867188,18.445004844665526,18.445004844665526,0.026939913540867878,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,2,40960,1,8192,32768.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,32768.0,40960.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.023615999147295952,0.03267199918627739,0.026281599886715412,0.0248800003901124,0.003167969318232417,3.1188158988952637,3.1837120056152344,3.133536005020142,3.133536005020142,0.018365009771617643,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,8192,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,8192.0,16384.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks16k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.02380800060927868,0.03283200040459633,0.025670399703085423,0.024639999493956566,0.0026095917641781453,5.1729278564453125,5.243135929107666,5.192828798294067,5.192828798294067,0.01984737475315108,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,8192,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,16384.0,24576.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks24k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.024032000452280045,0.03167999908328056,0.02665280010551214,0.02550400048494339,0.002510131946267113,7.223167896270752,7.257152080535889,7.231532812118529,7.231532812118529,0.009418898846297825,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,8192,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,24576.0,32768.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks32k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.01696000061929226,0.023679999634623528,0.01866880003362894,0.017455999739468098,0.002277860345236006,1.478559970855713,1.496127963066101,1.48257919549942,1.48257919549942,0.005278358037488471,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,4096,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,8192.0,12288.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks12k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.017023999243974686,0.030688000842928886,0.02055360022932291,0.018240000121295452,0.004065815389665311,2.651711940765381,2.6691839694976807,2.657548785209656,2.657548785209656,0.005897264516465086,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,4096,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,16384.0,20480.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks20k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.016767999157309532,0.024351999163627625,0.01840319987386465,0.017167999409139156,0.0025234367788448957,3.823551893234253,3.8631999492645264,3.8335999727249135,3.8335999727249135,0.011858923917429712,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,4096,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[4096],4096,4096,4096,4096.0,True,0.0,0.0,0.0,True,24576.0,28672.0,4096,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q4ks28k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.013952000066637993,0.03622400015592575,0.018214400112628936,0.014719999860972166,0.007277897466521675,0.7039039731025696,0.7163199782371521,0.7077983975410462,0.7077983975410462,0.004048424224033295,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,2048,8192.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,8192.0,10240.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks10k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.015104000456631184,0.03097599931061268,0.020870400313287973,0.02054399996995926,0.005422197456220526,1.2929600477218628,1.313088059425354,1.3006752014160157,1.3006752014160157,0.00790622446009414,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,2048,16384.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,16384.0,18432.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks18k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.014655999839305878,0.03232000023126602,0.019132800027728082,0.01673599984496832,0.0050857949387988245,1.8775999546051025,1.9125759601593018,1.8867840051651,1.8867840051651,0.01196043526228758,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,2048,24576.0,True,FLASH_ATTN,False,vllm020_batch_spec,[2048],2048,2048,2048,2048.0,True,0.0,0.0,0.0,True,24576.0,26624.0,2048,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q2ks26k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median
+0.0,0.0,0.0,0.0,0.0,0.023584000766277313,0.03126399964094162,0.02633600030094385,0.025200000032782555,0.0025710957466677864,9.283391952514648,9.48249626159668,9.331705665588379,9.331705665588379,0.06366970422148335,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2048,32,4,16,4,40960,1,8192,32768.0,True,FLASH_ATTN,False,vllm020_batch_spec,[8192],8192,8192,8192,8192.0,True,0.0,0.0,0.0,True,32768.0,40960.0,8192,BF16,generic,none,CUDA_EVENT,False,,,,,,,,,,,q8ks40k,measured_FA3_core_plus_measured_KV;reshape_assumed_zero;mean_as_median

runs/frontier-prefill-kvgrowth-fix-v0/profiles/profile-v5-kvgrowth/linear_op.csv

@@ -0,0 +1,37 @@
+time_stats.emb.min,time_stats.emb.max,time_stats.emb.mean,time_stats.emb.median,time_stats.emb.std,time_stats.input_layernorm.min,time_stats.input_layernorm.max,time_stats.input_layernorm.mean,time_stats.input_layernorm.median,time_stats.input_layernorm.std,time_stats.attn_pre_proj.min,time_stats.attn_pre_proj.max,time_stats.attn_pre_proj.mean,time_stats.attn_pre_proj.median,time_stats.attn_pre_proj.std,time_stats.attn_rope.min,time_stats.attn_rope.max,time_stats.attn_rope.mean,time_stats.attn_rope.median,time_stats.attn_rope.std,time_stats.attn_post_proj.min,time_stats.attn_post_proj.max,time_stats.attn_post_proj.mean,time_stats.attn_post_proj.median,time_stats.attn_post_proj.std,time_stats.post_attention_layernorm.min,time_stats.post_attention_layernorm.max,time_stats.post_attention_layernorm.mean,time_stats.post_attention_layernorm.median,time_stats.post_attention_layernorm.std,n_head,n_kv_head,n_embd,n_expanded_embd,vocab_size,use_gated_mlp,use_qk_norm,attn_output_gate,num_tokens,num_tensor_parallel_workers,padded_n_embd,padded_n_expanded_embd,model_arch,is_step2_mini,share_expert_dim,share_q_dim,measurement_type,profiling_precision,quant_signature
+0.029184000566601753,0.06780800223350525,0.03157280012965202,0.030736000277101994,0.005874173435341216,0.033215999603271484,0.04825599864125252,0.03443359974771738,0.0337119996547699,0.0031825678429048183,1.438431978225708,1.505568027496338,1.446228802204132,1.4429279565811157,0.014128607642643025,0.538752019405365,0.5440319776535034,0.5416463971138,0.5420799851417542,0.0016099306164432847,1.0073280334472656,1.0163840055465698,1.0098415970802308,1.0081279873847961,0.0032980457197329728,0.04022400081157684,0.041728001087903976,0.04091359991580248,0.04081599973142147,0.0004728505416767937,32,4,2048,768,151936,True,True,False,8192,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.01360000018030405,0.06784000247716904,0.017755200061947106,0.0179840000346303,0.00837681421401443,0.01836800016462803,0.03651199862360954,0.019606399815529585,0.018719999119639397,0.0038821958848767424,0.7512000203132629,0.8208960294723511,0.7566704005002975,0.75382399559021,0.014793092586577971,0.28995200991630554,0.29337599873542786,0.29135999977588656,0.29150401055812836,0.000998381071258815,0.5149760246276855,0.5169600248336792,0.5160208016633987,0.5158880054950714,0.0005038652783291977,0.02051199972629547,0.021503999829292297,0.021067200042307378,0.021104000508785248,0.00023542855306902367,32,4,2048,768,151936,True,True,False,4096,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.0080960001796484,0.04281599819660187,0.011092799971811474,0.011039999779313803,0.005489135752949302,0.012223999947309494,0.026335999369621277,0.013187199970707298,0.01247999956831336,0.0030236510562153375,0.3928639888763428,0.45372799038887024,0.3976895987987518,0.39528000354766846,0.012905176716136006,0.1547199934720993,0.15884800255298615,0.15712319910526276,0.1573439985513687,0.001165121945135037,0.26633599400520325,0.268095999956131,0.26719200164079665,0.2671840041875839,0.0004242740199415328,0.013024000450968742,0.013887999579310417,0.013489600038155913,0.013520000036805868,0.0002382817554057738,32,4,2048,768,151936,True,True,False,2048,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.00825599953532219,0.04755200073122978,0.02398160002194345,0.01961600035429001,0.00874683840888523,0.018880000337958336,0.03868800029158592,0.021590400114655496,0.0208320003002882,0.004057015751746236,0.24316799640655518,0.2710399925708771,0.2521967992186546,0.25065599381923676,0.007998418528894075,0.09644799679517746,0.19120000302791595,0.10407840013504029,0.09963199868798256,0.020043722414992166,0.13600000739097595,0.18892799317836761,0.15760480016469955,0.15760000050067902,0.0112223677907762,0.009664000011980534,0.010015999898314476,0.009836799977347255,0.009824000298976898,9.016971952948177e-05,32,4,2048,768,151936,True,True,False,1024,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.017376000061631203,0.044704001396894455,0.026318399980664254,0.027312000282108784,0.007565344034680866,0.01836800016462803,0.03014400042593479,0.021276800055056812,0.020655999891459942,0.002632977855097951,0.1438719928264618,0.17132799327373505,0.152497598528862,0.15012799948453903,0.007947150319625347,0.1430719941854477,0.19305600225925446,0.16630879789590836,0.1685439944267273,0.014628024163894684,0.08899199962615967,0.10467199981212616,0.0943599995225668,0.09374399855732918,0.003472669494074113,0.007615999784320593,0.007935999892652035,0.007769599952735007,0.0077760000713169575,7.680004540222077e-05,32,4,2048,768,151936,True,True,False,512,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.016992000862956047,0.0544000007212162,0.02573199989274144,0.026016000658273697,0.00803323401720434,0.018432000651955605,0.02348800003528595,0.020648000109940768,0.02062400057911873,0.0013939985047930988,0.10220800340175629,0.1361600011587143,0.11850560046732425,0.11684799939393997,0.010637754898360304,0.16710400581359863,0.21110400557518005,0.19078560024499894,0.19409599900245667,0.013714352646558832,0.06265600025653839,0.0740479975938797,0.06842879951000214,0.0690080001950264,0.0031292240446560557,0.00979200005531311,0.033440001308918,0.01864320016466081,0.017280000261962414,0.006957998188876383,32,4,2048,768,151936,True,True,False,256,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.017152000218629837,0.04396799951791763,0.025907999789342284,0.02598400041460991,0.007714554737915267,0.018400000408291817,0.03667199984192848,0.024132800102233887,0.02112000063061714,0.006120348733354326,0.10678400099277496,0.1363839954137802,0.1193264003843069,0.11583999916911125,0.009838647443214228,0.17017599940299988,0.22748799622058868,0.18853759989142418,0.18433599919080734,0.015728078443174653,0.04569600149989128,0.06652799993753433,0.05192639995366335,0.05151999928057194,0.004516605694976449,0.021856000646948814,0.026335999369621277,0.02384479995816946,0.023599999956786633,0.0011301153013314744,32,4,2048,768,151936,True,True,False,128,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.017343999817967415,1.0683200359344482,0.05748240072280168,0.029504000209271908,0.16366824814254827,0.018688000738620758,0.3317759931087494,0.03685439983382821,0.021151999942958355,0.06767422466731164,0.10255999863147736,0.9434880018234253,0.16412640027701855,0.11956800147891045,0.17964606281728834,0.1714559942483902,2.1306240558624268,0.3011296011507511,0.1926399990916252,0.42310127734378766,0.03574400022625923,0.6859520077705383,0.08389280084520578,0.04279999993741512,0.14321647071615612,0.020479999482631683,0.1831360012292862,0.033024000097066165,0.023856000043451786,0.03470987082429836,32,4,2048,768,151936,True,True,False,64,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.016095999628305435,0.05142400041222572,0.026363200135529043,0.02676799986511469,0.008637725852473854,0.01849599927663803,0.03577600046992302,0.022193600237369538,0.020848000422120094,0.004615266181181815,0.10540799796581268,0.15014399588108063,0.12211520001292228,0.11896000057458878,0.012772013396624768,0.17315199971199036,0.21478399634361267,0.1881632000207901,0.1873439997434616,0.011761164657572015,0.03481600061058998,0.058079998940229416,0.042200000025331974,0.03969600051641464,0.006461281521769989,0.02143999934196472,0.038816001266241074,0.02466559996828437,0.023856000043451786,0.0035418033748569927,32,4,2048,768,151936,True,True,False,32,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.015904000028967857,0.03753599897027016,0.023127999808639287,0.02527999971061945,0.0054230027890305385,0.018400000408291817,0.03001599945127964,0.020488000102341176,0.020096000283956528,0.002585688394137697,0.10355199873447418,0.1438400000333786,0.11536479964852334,0.11124800145626068,0.011136028294919255,0.16502399742603302,0.2072959989309311,0.18646399974822997,0.19075199961662292,0.013296437761247597,0.03142400085926056,0.05215999856591225,0.03888959977775812,0.03750399872660637,0.0057399302067536314,0.020128000527620316,0.04064000025391579,0.023937600292265417,0.023648000322282314,0.004144197400898248,32,4,2048,768,151936,True,True,False,16,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.015200000256299973,0.037151999771595,0.02144480012357235,0.02195199951529503,0.005576364091933697,0.017952000722289085,0.0226879995316267,0.019934400077909233,0.019952000118792057,0.0011812499470458758,0.10063999891281128,0.13468800485134125,0.11595199964940547,0.1207519993185997,0.011974301621092394,0.16332800686359406,0.20748800039291382,0.18162400051951408,0.1767839938402176,0.01474120743720334,0.03222399950027466,0.043455999344587326,0.03829439990222454,0.03859200142323971,0.0031378131240041122,0.020959999412298203,0.03587200120091438,0.023795200139284135,0.023423999547958374,0.0031339489695198443,32,4,2048,768,151936,True,True,False,8,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+0.014944000169634819,0.04022400081157684,0.02162720002233982,0.022463999688625336,0.006017219317026815,0.018464000895619392,0.026528000831604004,0.021264000236988066,0.020896000787615776,0.001934095017586082,0.10156799852848053,0.1703999936580658,0.12565439902245998,0.1244799979031086,0.01641776722785409,0.1653759926557541,0.23865599930286407,0.1969360001385212,0.19223999977111816,0.02058903050274278,0.03254399821162224,0.06752000004053116,0.0443536002188921,0.041519999504089355,0.00984829071098989,0.020096000283956528,0.040031999349594116,0.026934400014579297,0.02478400059044361,0.00562071702648593,32,4,2048,768,151936,True,True,False,1,1,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.7531200051307678,0.8461440205574036,0.7618160009384155,0.7576479911804199,0.019464233617982506,0.2922559976577759,0.2985599935054779,0.2953856036067009,0.29576000571250916,0.001575901077689139,0.510047972202301,0.5140479803085327,0.5121696025133133,0.5123839974403381,0.0012263137313476844,,,,,,32,4,2048,768,151936,True,True,False,8192,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.39529600739479065,0.47279998660087585,0.40216960161924364,0.39825600385665894,0.0162749796240819,0.15625600516796112,0.16211199760437012,0.15959519892930984,0.15988799929618835,0.0011493418942396922,0.2635200023651123,0.2642880082130432,0.2639120012521744,0.26392000913619995,0.0001903593401384135,,,,,,32,4,2048,768,151936,True,True,False,4096,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.2531839907169342,0.32419198751449585,0.26924319565296173,0.26049599051475525,0.01657194262368116,0.10051199793815613,0.2375359982252121,0.12411200068891048,0.10311999917030334,0.039466165428540506,0.15881599485874176,0.19289599359035492,0.16896959990262986,0.1640480011701584,0.009162416086418127,,,,,,32,4,2048,768,151936,True,True,False,2048,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.15302400290966034,0.1780800074338913,0.16284480094909667,0.16113600134849548,0.00740355661356144,0.14521600306034088,0.20233599841594696,0.17807039842009545,0.1796799972653389,0.013908448560094403,0.0907519981265068,0.09750399738550186,0.09460479989647866,0.09478399902582169,0.0017965487667361475,,,,,,32,4,2048,768,151936,True,True,False,1024,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11740799993276596,0.17958399653434753,0.13262080028653145,0.12878400087356567,0.014146060532423056,0.17468799650669098,0.21161599457263947,0.1910431995987892,0.18966399878263474,0.01270903647700971,0.05926400050520897,0.07577600330114365,0.06530559975653887,0.06404799968004227,0.00479458462514625,,,,,,32,4,2048,768,151936,True,True,False,512,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11260800063610077,0.17209599912166595,0.13610880002379416,0.13809599727392197,0.017675922458993677,0.18729600310325623,0.26822400093078613,0.20815680101513861,0.2078079953789711,0.01700718593658771,0.04499199986457825,0.06537599861621857,0.0507551996037364,0.04787199944257736,0.005727123232692158,,,,,,32,4,2048,768,151936,True,True,False,256,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11257600039243698,0.23452800512313843,0.13866880126297473,0.13964799791574478,0.026590047697062115,0.1828799992799759,0.24751999974250793,0.20735519900918006,0.20670399814844131,0.01500438131586336,0.03654399886727333,0.05593600124120712,0.0408239996060729,0.03892800025641918,0.004864409450710354,,,,,,32,4,2048,768,151936,True,True,False,128,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11734399944543839,0.18726399540901184,0.13890240006148816,0.13308800011873245,0.019086167340006257,0.16991999745368958,0.2443840056657791,0.19185120090842248,0.19257599860429764,0.018017536159219673,0.03033600002527237,0.04956800118088722,0.038387199863791466,0.03750400058925152,0.00500897018702887,,,,,,32,4,2048,768,151936,True,True,False,64,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11420799791812897,0.18115200102329254,0.13938880078494548,0.1393439993262291,0.018141313962922536,0.1693439930677414,0.221343994140625,0.19187839925289155,0.19438399374485016,0.01430752121272052,0.03017600066959858,0.06019200012087822,0.03866560012102127,0.0364960003644228,0.0074199790031205266,,,,,,32,4,2048,768,151936,True,True,False,32,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11276800185441971,0.16223999857902527,0.1331360016018152,0.13305599987506866,0.01432493740801146,0.17187200486660004,0.23625600337982178,0.19287680014967917,0.1913280040025711,0.01754628831589704,0.029823999851942062,0.07574400305747986,0.03900959976017475,0.036927999928593636,0.009277465083962879,,,,,,32,4,2048,768,151936,True,True,False,16,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.1130559965968132,0.19814400374889374,0.13379519879817964,0.12531199678778648,0.020074427302248836,0.16841599345207214,0.21139200031757355,0.1893615983426571,0.19092799723148346,0.012418720676762711,0.029503999277949333,0.07558400183916092,0.040144000016152856,0.03728000074625015,0.010223239196498205,,,,,,32,4,2048,768,151936,True,True,False,8,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.115167997777462,0.2640640139579773,0.13805920109152794,0.13118399679660797,0.03132849683515479,0.17103999853134155,0.2977280020713806,0.19899839907884598,0.1976960003376007,0.028928046763067948,0.0297279991209507,0.05990400165319443,0.03834720011800528,0.0363520011305809,0.0070885330713495905,,,,,,32,4,2048,768,151936,True,True,False,1,2,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.3940800130367279,0.46483200788497925,0.39923040121793746,0.3957759886980057,0.015108903906233569,0.16022400557994843,0.1634880006313324,0.16203359961509706,0.16228799521923065,0.0010220720912414007,0.26073598861694336,0.2627840042114258,0.26138080209493636,0.2613760083913803,0.00041640363494172775,,,,,,32,4,2048,768,151936,True,True,False,8192,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.2536959946155548,0.27161601185798645,0.25960480123758317,0.2577280104160309,0.005142017404245015,0.09849599748849869,0.10281600058078766,0.10057279989123344,0.10063999891281128,0.0008836232237268523,0.13913600146770477,0.17606399953365326,0.15875840038061143,0.15988799929618835,0.010058952112389172,,,,,,32,4,2048,768,151936,True,True,False,4096,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.15142400562763214,0.1780479997396469,0.16139679849147798,0.1602879986166954,0.0070656055731385115,0.14601600170135498,0.23343999683856964,0.17913119941949845,0.179967999458313,0.023065274309176566,0.09388799965381622,0.12108799815177917,0.10317599996924401,0.10073599964380264,0.007364345618470178,,,,,,32,4,2048,768,151936,True,True,False,2048,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.12198399752378464,0.18406400084495544,0.13770400024950505,0.13232000172138214,0.01623164885687898,0.1773120015859604,0.20688000321388245,0.1870912007987499,0.18535999953746796,0.00825628058448234,0.05766399949789047,0.06739199906587601,0.06187200043350458,0.061216000467538834,0.003004312467037863,,,,,,32,4,2048,768,151936,True,True,False,1024,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11737599968910217,0.18614399433135986,0.13109439946711063,0.12771200388669968,0.01372168725934724,0.17587199807167053,0.2699519991874695,0.19926720038056372,0.19075199961662292,0.02327478982490538,0.041728001087903976,0.06224000081419945,0.05000480003654957,0.049375999718904495,0.006185850944273961,,,,,,32,4,2048,768,151936,True,True,False,512,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.1159679964184761,0.1610880047082901,0.12988320142030715,0.12494400516152382,0.011865375783184065,0.1767680048942566,0.27379199862480164,0.21035519987344742,0.21275199949741364,0.023206964017550125,0.037087999284267426,0.062144000083208084,0.04439679980278015,0.04283200018107891,0.006155226392511492,,,,,,32,4,2048,768,151936,True,True,False,256,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.12015999853610992,0.158720001578331,0.13573280088603495,0.1343199983239174,0.010666830836014414,0.1737920045852661,0.22127999365329742,0.20261440128087999,0.20321600139141083,0.011153965849067301,0.03587200120091438,0.04944000020623207,0.038265600241720675,0.037328001111745834,0.0030431580113575636,,,,,,32,4,2048,768,151936,True,True,False,128,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.1212799996137619,0.15139199793338776,0.13183839991688728,0.13014400005340576,0.00851207547022104,0.17257599532604218,0.2250880002975464,0.1872655987739563,0.18193599581718445,0.013638284975248818,0.02969600073993206,0.0525440014898777,0.03840640028938651,0.03444799967110157,0.007598511754254174,,,,,,32,4,2048,768,151936,True,True,False,64,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11270400136709213,0.1624639928340912,0.13508000001311302,0.13180799782276154,0.014243564451606775,0.17052799463272095,0.23715199530124664,0.1953311987221241,0.1966560035943985,0.01500670718978398,0.030239999294281006,0.05270399898290634,0.03807039987295866,0.03742399998009205,0.004906165602021684,,,,,,32,4,2048,768,151936,True,True,False,32,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.12319999933242798,0.17919999361038208,0.14059039913117885,0.14156799763441086,0.016217662982107223,0.1701119989156723,0.21987199783325195,0.18903039917349815,0.1870879977941513,0.014117854507841239,0.030368000268936157,0.06406400352716446,0.04045119984075427,0.03710399940609932,0.00878942205983628,,,,,,32,4,2048,768,151936,True,True,False,16,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11296000331640244,0.16761599481105804,0.13843199908733367,0.13814399391412735,0.014925286935582404,0.1711679995059967,0.21561600267887115,0.1906527981162071,0.1876479983329773,0.011803992622137974,0.0306560005992651,0.09216000139713287,0.041129599791020155,0.036847999319434166,0.013739721468154687,,,,,,32,4,2048,768,151936,True,True,False,8,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none
+,,,,,,,,,,0.11929599940776825,0.2008959949016571,0.14535359852015972,0.13971199840307236,0.024769473061576282,0.16803200542926788,0.25123199820518494,0.19573760256171227,0.19366399943828583,0.023196011081705884,0.03049599938094616,0.08089599758386612,0.04739360017701984,0.0453919991850853,0.01286389846889463,,,,,,32,4,2048,768,151936,True,True,False,1,4,2048,768,generic,False,,,CUDA_EVENT,BF16,none

runs/frontier-prefill-kvgrowth-fix-v0/profiles/profile-v5-kvgrowth/manifest.json

@@ -0,0 +1,20 @@
+{
+  "schema": "frontier-profile-v5-kvgrowth-v1",
+  "base": "runs/frontier-split-rootcause-v0/frozen-inputs/q30-profiles/profile-v4-trace-final/attention.csv",
+  "base_sha256": "4f0c42d2c47f52880d81b8d2b2d7d52bd62d2c34e0f3a7f94c4c6cbd64d751c8",
+  "raw_inputs": {
+    "runs/frontier-prefill-kvgrowth-fix-v0/fleet-artifacts/kvgrowth-attn-grid-tp1-20260723-v1-20260722T175305475419Z/artifacts/outputs/kvgrowth/tp1/raw/flashattn-kvgrowth-tp1.json": "f2e419e94a2de36e9d106acdb677b2e8f8ad761d8c467b2995f308981be3898a",
+    "runs/frontier-prefill-kvgrowth-fix-v0/fleet-artifacts/kvgrowth-attn-grid-tp2-20260723-v1-20260722T175300181578Z/artifacts/outputs/kvgrowth/tp2/raw/flashattn-kvgrowth-tp2.json": "aaad5c653b94e50f0797401c06470972c4243122a9517b638635b1e9d322d531",
+    "runs/frontier-prefill-kvgrowth-fix-v0/fleet-artifacts/kvgrowth-attn-grid-tp4-20260723-v1-20260722T175255849269Z/artifacts/outputs/kvgrowth/tp4/raw/flashattn-kvgrowth-tp4.json": "177ce3ccf883b8d5ac763e671e27b778de2b627ef72da66c75ca6c94532c1efc"
+  },
+  "appended_rows": 30,
+  "anchor_checks": [
+    "anchor q1ks8k/TP1: v4=1.1445ms new=1.1469ms rel_diff=0.2%",
+    "anchor q512s4k/TP1: v4=0.3391ms new=0.3348ms rel_diff=1.3%",
+    "anchor q1ks8k/TP2: v4=0.6126ms new=0.6122ms rel_diff=0.1%",
+    "anchor q512s4k/TP2: v4=0.2086ms new=0.2078ms rel_diff=0.4%",
+    "anchor q1ks8k/TP4: v4=0.3445ms new=0.3552ms rel_diff=3.1%",
+    "anchor q512s4k/TP4: v4=0.1536ms new=0.1551ms rel_diff=1.0%"
+  ],
+  "output_sha256": "ff32c38975e68565c85770d85b92bc310ab1d8e34fd02327bef7fb305a9ffae3"
+}

runs/frontier-prefill-kvgrowth-fix-v0/profiles/profile-v5-kvgrowth/moe.csv

@@ -0,0 +1,73 @@
+time_stats.moe_gating_linear.min,time_stats.moe_gating_linear.max,time_stats.moe_gating_linear.mean,time_stats.moe_gating_linear.median,time_stats.moe_gating_linear.std,time_stats.moe_gating_routing_topk.min,time_stats.moe_gating_routing_topk.max,time_stats.moe_gating_routing_topk.mean,time_stats.moe_gating_routing_topk.median,time_stats.moe_gating_routing_topk.std,time_stats.moe_shuffling.min,time_stats.moe_shuffling.max,time_stats.moe_shuffling.mean,time_stats.moe_shuffling.median,time_stats.moe_shuffling.std,time_stats.moe_grouped_gemm.min,time_stats.moe_grouped_gemm.max,time_stats.moe_grouped_gemm.mean,time_stats.moe_grouped_gemm.median,time_stats.moe_grouped_gemm.std,num_tokens,num_experts,num_experts_per_device,expert_parallel_size,routing_runtime_path,routing_assignment_policy,routing_weight_policy,routing_uses_router_logits,gating_runtime_context,gating_runtime_context_impl,router_topk,hidden_dim,expert_hidden_dim,use_gated,num_tensor_parallel_workers,total_routed_tokens,model_expansion_ratio,tokens_per_expert_avg,tokens_to_experts_ratio,expert_utilization,min_load_ratio,load_imbalance_cv,max_load_ratio,load_entropy,load_gini_coefficient,load_distribution,seed,moe_grouped_gemm_backend,measurement_type,profiling_precision,model_arch,quant_signature,router_median_nonadditivity_ratio,projection_policy
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.33926400542259216,0.405023992061615,0.36780479848384856,0.36507199704647064,0.01690507644474779,1,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,1.1943039894104004,1.286784052848816,1.228384006023407,1.2273280024528503,0.02832547242381263,8,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,64,0.375,0.5,0.5,0.3984375,0.0,1.346291201783626,4.0,5.59375,0.661865234375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,1.679744005203247,1.766144037246704,1.7095808148384095,1.7015680074691772,0.02438921262998535,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,128,0.375,1.0,1.0,0.625,0.0,1.015504800579495,5.0,6.15516433212955,0.529052734375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,2.1576640605926514,2.2921600341796875,2.2097824096679686,2.188944101333618,0.045572321842012986,32,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,256,0.375,2.0,2.0,0.875,0.0,0.6343057228182637,2.5,6.64370748444639,0.35369873046875,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,2.422368049621582,2.5130879878997803,2.4516672134399413,2.434159994125366,0.03278900287381846,64,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,512,0.375,4.0,4.0,0.984375,0.0,0.4921254921257382,2.25,6.817190042344769,0.272369384765625,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,2.2217600345611572,2.289599895477295,2.2571327924728393,2.263375997543335,0.021660416089449488,128,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,1024,0.375,8.0,8.0,1.0,0.125,0.3486861500690843,1.875,6.908192310183997,0.197662353515625,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,2.18668794631958,2.318079948425293,2.2218016147613526,2.211087942123413,0.035380897213135316,256,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,2048,0.375,16.0,16.0,1.0,0.4375,0.2525504668006971,1.875,6.953347743053017,0.1410369873046875,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,2.2291839122772217,2.3929600715637207,2.2908096313476562,2.2804640531539917,0.04479348924786221,512,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,4096,0.375,32.0,32.0,1.0,0.65625,0.15765965680164504,1.5625,6.98229848728205,0.08779525756835938,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,2.0678720474243164,2.1297600269317627,2.0837119817733765,2.0779199600219727,0.017880044357986735,1024,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,8192,0.375,64.0,64.0,1.0,0.625,0.12169081635504074,1.3125,6.9892029662356325,0.06879425048828125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,2.916032075881958,3.0819520950317383,2.9805248022079467,2.9656319618225098,0.05482195799019572,2048,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,16384,0.375,128.0,128.0,1.0,0.796875,0.07935434147688751,1.1796875,6.9954297964750305,0.044734954833984375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,4.421599864959717,4.535359859466553,4.486294317245483,4.497056007385254,0.036990243787549344,4096,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,32768,0.375,256.0,256.0,1.0,0.8203125,0.060849326483103046,1.17578125,6.9973188375859685,0.033740997314453125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,7.302591800689697,7.402751922607422,7.354758310317993,7.3464319705963135,0.032142662400335566,8192,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,65536,0.375,512.0,512.0,1.0,0.890625,0.0412323087266341,1.08984375,6.998772433185578,0.02334284782409668,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.35280001163482666,0.39692801237106323,0.37662720382213594,0.37196800112724304,0.013401318050665304,1,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,0.4692479968070984,0.5523840188980103,0.5134752035140991,0.5100640058517456,0.02291433464135784,8,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,64,0.375,0.5,0.5,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,0.34652799367904663,0.4119040071964264,0.3789471983909607,0.38550400733947754,0.02073945105803335,16,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,128,0.375,1.0,1.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,0.31462401151657104,0.7456960082054138,0.38617280423641204,0.34545600414276123,0.12230201266253077,32,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,256,0.375,2.0,2.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,0.32521599531173706,0.419871985912323,0.36325119733810424,0.34968000650405884,0.03161798848223672,64,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,512,0.375,4.0,4.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,0.289792001247406,0.4663360118865967,0.4091839998960495,0.41655999422073364,0.0446001986506615,128,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,1024,0.375,8.0,8.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,0.3761279881000519,0.4416320025920868,0.40686399936676027,0.40540799498558044,0.02257778769899645,256,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,2048,0.375,16.0,16.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,0.7172480225563049,0.8663039803504944,0.7723807990550995,0.7591840028762817,0.04164772379451242,512,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,4096,0.375,32.0,32.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,1.0195519924163818,1.2216639518737793,1.1253888130187988,1.1453600525856018,0.06548005322243594,1024,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,8192,0.375,64.0,64.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,1.7490559816360474,1.9644800424575806,1.8529024004936219,1.814303994178772,0.08042565617327288,2048,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,16384,0.375,128.0,128.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,3.2479360103607178,3.385279893875122,3.296070408821106,3.2800960540771484,0.04525529026442046,4096,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,32768,0.375,256.0,256.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,6.344799995422363,6.517856121063232,6.464438438415527,6.478623867034912,0.05116674443145098,8192,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,1,65536,0.375,512.0,512.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.2648000121116638,0.325439989566803,0.28852800130844114,0.28390398621559143,0.01933778377635077,1,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,0.7347840070724487,0.862496018409729,0.7769344031810761,0.769216001033783,0.03290485328796285,8,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,64,0.375,0.5,0.5,0.421875,0.0,1.346291201783626,6.0,5.652114648336087,0.636962890625,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,0.9198399782180786,0.9646080136299133,0.9412063956260681,0.9411839842796326,0.014939365085478117,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,128,0.375,1.0,1.0,0.5703125,0.0,1.118033988749895,5.0,6.008641773518898,0.580810546875,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,1.2796800136566162,1.3484159708023071,1.3006976008415223,1.2929120063781738,0.020807998177176254,32,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,256,0.375,2.0,2.0,0.8828125,0.0,0.6959705453537527,3.0,6.60872850615583,0.38055419921875,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,1.3630399703979492,1.4430400133132935,1.3909215927124023,1.3892319798469543,0.022335744492366926,64,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,512,0.375,4.0,4.0,0.984375,0.0,0.5201036555341637,3.0,6.798826509158851,0.28302001953125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,1.27948796749115,1.3904000520706177,1.3176063895225525,1.309440016746521,0.038060887827312775,128,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,1024,0.375,8.0,8.0,1.0,0.25,0.3511282039725661,1.875,6.91002266305238,0.1970977783203125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,1.264415979385376,1.3145920038223267,1.2791999936103822,1.2753440141677856,0.014130605249568332,256,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,2048,0.375,16.0,16.0,1.0,0.375,0.24692938483248605,1.6875,6.955481130775285,0.13909912109375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,1.3081920146942139,1.347648024559021,1.3292255997657776,1.329967975616455,0.014558863679016933,512,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,4096,0.375,32.0,32.0,1.0,0.625,0.17143053326165383,1.5625,6.9786675275754035,0.09520339965820312,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,1.242751955986023,1.3112000226974487,1.2747935891151427,1.266207993030548,0.021093073517695057,1024,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,8192,0.375,64.0,64.0,1.0,0.78125,0.11000099875256815,1.296875,6.991308871213679,0.062183380126953125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,1.7388160228729248,1.8077759742736816,1.772764801979065,1.772704005241394,0.021056644077284283,2048,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,16384,0.375,128.0,128.0,1.0,0.78125,0.0864630150197678,1.1796875,6.994552526394139,0.048796653747558594,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,2.6563520431518555,2.7063679695129395,2.6785055875778196,2.6791679859161377,0.01639963463052944,4096,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,32768,0.375,256.0,256.0,1.0,0.8671875,0.06127686514721937,1.16015625,6.997291583027146,0.03497934341430664,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,4.386879920959473,4.452256202697754,4.4108480453491214,4.406303882598877,0.019768161791937636,8192,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,65536,0.375,512.0,512.0,1.0,0.884765625,0.041723768525324195,1.1171875,6.998746434318934,0.02298593521118164,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.24208000302314758,0.4028480052947998,0.3041536003351212,0.277103990316391,0.05660721484881584,1,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,0.2447039932012558,0.30502399802207947,0.26446720361709597,0.26265600323677063,0.016744548364435372,8,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,64,0.375,0.5,0.5,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,0.2337920069694519,0.2881599962711334,0.26074880361557007,0.264384001493454,0.016469850143940968,16,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,128,0.375,1.0,1.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,0.23369599878787994,0.28591999411582947,0.25465920120477675,0.25385600328445435,0.01593204652619194,32,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,256,0.375,2.0,2.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,0.2295999974012375,0.26556798815727234,0.24674240052700042,0.2497600018978119,0.010345732066199003,64,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,512,0.375,4.0,4.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,0.21721599996089935,0.29020801186561584,0.2394208014011383,0.2346400022506714,0.018747330357768585,128,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,1024,0.375,8.0,8.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,0.2717759907245636,0.305184006690979,0.28813759982585907,0.28809599578380585,0.01183422600545854,256,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,2048,0.375,16.0,16.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,0.3917759954929352,0.43772798776626587,0.41130879521369934,0.4131519943475723,0.012992473640805227,512,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,4096,0.375,32.0,32.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,0.6176639795303345,0.7009919881820679,0.642767995595932,0.6330719888210297,0.024074084919938756,1024,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,8192,0.375,64.0,64.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,1.0820800065994263,1.1674879789352417,1.1034304022789,1.0977439880371094,0.0233067292981566,2048,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,16384,0.375,128.0,128.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,1.9809919595718384,2.0415360927581787,2.003715181350708,1.992751955986023,0.022066076434645737,4096,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,32768,0.375,256.0,256.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,3.790112018585205,3.8651199340820312,3.829139161109924,3.8230879306793213,0.025177464160110564,8192,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,2,65536,0.375,512.0,512.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.212351992726326,0.24383999407291412,0.22760000079870224,0.22723200172185898,0.01050568575837594,1,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,0.47494399547576904,0.5184000134468079,0.4920704007148743,0.49169600009918213,0.011991064701471855,8,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,64,0.375,0.5,0.5,0.3984375,0.0,1.3919410907075054,6.0,5.570159765557392,0.667236328125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,0.6360960006713867,0.7004479765892029,0.6608384013175964,0.6572319865226746,0.020416877242438597,16,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,128,0.375,1.0,1.0,0.625,0.0,1.0307764064044151,4.0,6.138251855282827,0.5382080078125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,0.780896008014679,0.8301439881324768,0.80346559882164,0.8030399978160858,0.016801230312128875,32,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,256,0.375,2.0,2.0,0.859375,0.0,0.6903350635742038,3.0,6.5943747091218174,0.38067626953125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,0.8607040047645569,0.9195200204849243,0.8783008038997651,0.8751039803028107,0.01719059253115595,64,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,512,0.375,4.0,4.0,0.9765625,0.0,0.49410588440130926,2.75,6.814452474347134,0.271270751953125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,0.833952009677887,0.894752025604248,0.8619967997074127,0.863215982913971,0.018716378851797198,128,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,1024,0.375,8.0,8.0,1.0,0.25,0.33693529145074724,2.125,6.9186075263155535,0.1867218017578125,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,0.834879994392395,0.8871039748191833,0.8651552021503448,0.8638879954814911,0.015262430894400969,256,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,2048,0.375,16.0,16.0,1.0,0.375,0.25567294018677456,1.8125,6.952441049154937,0.14349365234375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,0.8518080115318298,0.9097599983215332,0.8810272097587586,0.8751040101051331,0.017005819633271906,512,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,4096,0.375,32.0,32.0,1.0,0.625,0.1747801353218523,1.53125,6.978069554482723,0.09820938110351562,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,0.8470079898834229,0.9010239839553833,0.8694015920162201,0.8716959953308105,0.016138881171190216,1024,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,8192,0.375,64.0,64.0,1.0,0.65625,0.1158122428154187,1.3125,6.9901908183358845,0.06445503234863281,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,1.1698240041732788,1.2311359643936157,1.1888479948043824,1.1890720129013062,0.017978797794492758,2048,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,16384,0.375,128.0,128.0,1.0,0.78125,0.08347181893108634,1.1796875,6.994921772573154,0.046871185302734375,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,1.7702720165252686,1.8097599744796753,1.7919103980064393,1.7956640124320984,0.012641295676021557,4096,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,32768,0.375,256.0,256.0,1.0,0.78125,0.06866734477822484,1.20703125,6.996602562938728,0.03801727294921875,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,2.968672037124634,3.0278079509735107,2.9899007797241213,2.9824799299240112,0.01816414122631667,8192,128,128,1,standard_fused_topk,logit_topk,softmax_renorm,True,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,65536,0.375,512.0,512.0,1.0,0.8984375,0.04399546833977376,1.126953125,6.998607314922362,0.024699926376342773,uniform_random_logits,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02502400055527687,0.06092799827456474,0.033839999698102474,0.028672000393271446,0.010315255343709818,0.019360000267624855,0.0352960005402565,0.023424000293016434,0.022064000368118286,0.0038898102289194572,0.0,0.0,0.0,0.0,0.0,0.19574399292469025,0.2512960135936737,0.21939200013875962,0.2199999988079071,0.017532156418212565,1,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,8,0.375,0.0625,0.0625,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0233364439829928,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025280000641942024,0.05132799968123436,0.030641599837690593,0.027343999594449997,0.007562409232763304,0.020160000771284103,0.052960000932216644,0.024145600199699403,0.021424000151455402,0.007450198645599985,0.0,0.0,0.0,0.0,0.0,0.20483200252056122,0.24006399512290955,0.22215040028095245,0.22433599829673767,0.00969639786132892,8,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,64,0.375,0.5,0.5,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9806430689981738,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024831999093294144,0.046560000628232956,0.030371200107038022,0.02763199992477894,0.005878487205028602,0.020320000126957893,0.04560000076889992,0.02601920012384653,0.02270400058478117,0.00751129965906799,0.0,0.0,0.0,0.0,0.0,0.20559999346733093,0.24726399779319763,0.22126719802618028,0.22207999974489212,0.01328093478485499,16,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,128,0.375,1.0,1.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9103623678483975,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024288000538945198,0.049375999718904495,0.03086080001667142,0.0267359996214509,0.0070864531384997225,0.020479999482631683,0.030912000685930252,0.02274719988927245,0.021359999664127827,0.0029966813650672505,0.0,0.0,0.0,0.0,0.0,0.20003199577331543,0.2301120012998581,0.21453119963407516,0.21598400175571442,0.010402239855151332,32,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,256,0.375,2.0,2.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9610778571819444,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02393599972128868,0.04342399910092354,0.029380799923092126,0.026688000187277794,0.0057374782000526574,0.020031999796628952,0.036607999354600906,0.022487999964505435,0.020911999978125095,0.0038135203062103235,0.0,0.0,0.0,0.0,0.0,0.19551999866962433,0.22972799837589264,0.21238719969987868,0.21488000452518463,0.010706095835489097,64,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,512,0.375,4.0,4.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9952941013961014,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.025407999753952026,0.05510399863123894,0.031430399790406224,0.02798399981111288,0.007335050273982725,0.020479999482631683,0.03561599925160408,0.02275839988142252,0.021551999263465405,0.003545718365165811,0.0,0.0,0.0,0.0,0.0,0.19420799612998962,0.2903999984264374,0.2211231991648674,0.21476799994707108,0.025955008886196004,128,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,1024,0.375,8.0,8.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9273256282883522,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.0244159996509552,0.05395200103521347,0.03188959984108806,0.026031999848783016,0.00943995927387483,0.02051199972629547,0.036607999354600906,0.023247999791055917,0.02147199958562851,0.003910623837069648,0.0,0.0,0.0,0.0,0.0,0.2290560007095337,0.289247989654541,0.2543327987194061,0.24939200282096863,0.01663236753503115,256,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,2048,0.375,16.0,16.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,1.0380599882396606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.024512000381946564,0.04620800167322159,0.02884640023112297,0.026320000179111958,0.005516557583355925,0.02054399996995926,0.03846399858593941,0.023401600029319524,0.021263999864459038,0.0044742944198265374,0.0,0.0,0.0,0.0,0.0,0.30831998586654663,0.36953601241111755,0.3324000000953674,0.3288639932870865,0.018617999572156707,512,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,4096,0.375,32.0,32.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9569603278386984,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02412799932062626,0.06940799951553345,0.030817600432783365,0.026800000108778477,0.010047085187652939,0.020128000527620316,0.044096000492572784,0.024606400076299904,0.022304000332951546,0.00622857166823714,0.0,0.0,0.0,0.0,0.0,0.462911993265152,0.5497919917106628,0.4893856018781662,0.4816960096359253,0.023348152887178286,1024,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,8192,0.375,64.0,64.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.9524274993623046,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.02831999957561493,0.043487999588251114,0.031744000129401685,0.02991999965161085,0.003973415836428909,0.02070399932563305,0.029343999922275543,0.022886400017887353,0.021743999794125557,0.0026873065045088873,0.0,0.0,0.0,0.0,0.0,0.7662079930305481,0.8717759847640991,0.788454395532608,0.7744799852371216,0.03174746482604812,2048,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,16384,0.375,128.0,128.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8971818172100244,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.03830400109291077,0.06268800050020218,0.043337599746882914,0.040511999279260635,0.005823016247946116,0.023135999217629433,0.03747199848294258,0.025206399988383053,0.02393599972128868,0.003271381256231161,0.0,0.0,0.0,0.0,0.0,1.363935947418213,1.4143040180206299,1.3812703967094422,1.3798720240592957,0.014770450075530007,4096,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,32768,0.375,256.0,256.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.8113207890716184,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize
+0.05660799890756607,0.07932800054550171,0.06249920018017292,0.06039999984204769,0.005636461691480845,0.02956799976527691,0.03747199848294258,0.031126399897038935,0.030287999659776688,0.002157571006631541,0.0,0.0,0.0,0.0,0.0,2.5507519245147705,2.680704116821289,2.579859209060669,2.566223978996277,0.03673283558365955,8192,128,128,1,standard_fused_topk,fixed_hotset8,softmax_renorm,False,standalone_legacy,vllm020_replicated_linear,8,2048,768,True,4,65536,0.375,512.0,512.0,0.0625,0.0,3.872983346207417,16.0,3.0,0.9375,hotset8,20260716,FlashInfer CUTLASS,CUDA_EVENT,BF16,generic,none,0.883909666885606,measured_gate+topk+modular_expert;shuffling_zero_because_expert_measurement_includes_prepare_finalize

runs/frontier-prefill-kvgrowth-fix-v0/run_replay_v5.py

@@ -0,0 +1,105 @@
+#!/usr/bin/env python3
+"""Replay an S3-real cell with profile-v5-kvgrowth (CPU, no GPU needed).
+
+Adapted from runs/frontier-s3-real-v0/run_frontier_prefix_replay.py with one
+change: the attention profile input is overridden to profile-v5-kvgrowth so the
+predictor retrains with the KV-context grid. Everything else (curves, trace,
+config argv, prefix caching) is identical to the S3-real sim runs, so old-vs-new
+differs by exactly one variable.
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import importlib.util
+import json
+import sys
+from pathlib import Path
+
+
+csv.field_size_limit(16 * 1024 * 1024)
+
+
+ROOT = Path(__file__).resolve().parent
+S3_REAL = ROOT.parent / "frontier-s3-real-v0"
+
+
+def load_s3_module():
+    spec = importlib.util.spec_from_file_location(
+        "s3_prefix_replay", S3_REAL / "run_frontier_prefix_replay.py"
+    )
+    module = importlib.util.module_from_spec(spec)
+    sys.path.insert(0, str(S3_REAL))  # trace_utils import
+    spec.loader.exec_module(module)
+    return module
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--trace", type=Path, required=True)
+    parser.add_argument("--output-root", type=Path, required=True)
+    parser.add_argument("--config", choices=("tp4_mns16", "tp2_mns16", "tp1_mns16"), required=True)
+    parser.add_argument("--label", required=True)
+    parser.add_argument("--max-tokens", type=int, required=True)
+    parser.add_argument("--duration-s", type=float)
+    parser.add_argument("--cache-root", type=Path)
+    parser.add_argument(
+        "--reuse-model-cache",
+        action="store_true",
+        help="Load predictor models from the content-addressed cache when available.",
+    )
+    parser.add_argument(
+        "--attention-profile",
+        type=Path,
+        default=ROOT / "profiles/profile-v5-kvgrowth/attention.csv",
+    )
+    args = parser.parse_args()
+
+    profile = args.attention_profile.resolve()
+    if not profile.is_file():
+        raise SystemExit(f"attention profile missing: {profile}")
+
+    module = load_s3_module()
+    original_replace = module.replace_flag
+
+    def replace_and_override(argv: list[str], flag: str, value: str) -> None:
+        original_replace(argv, flag, value)
+        # Piggyback on the first replace_flag call (trace file) to inject the
+        # profile override exactly once per run.
+        atten_flag = "--random_forrest_execution_time_predictor_config_atten_input_file"
+        if flag.endswith("trace_file") and atten_flag in argv:
+            original_replace(argv, atten_flag, str(profile))
+            no_cache_flag = "--random_forrest_execution_time_predictor_config_no_cache"
+            if args.reuse_model_cache and no_cache_flag in argv:
+                argv.remove(no_cache_flag)
+
+    module.replace_flag = replace_and_override
+    # The shared S3 parser predates TP1; reuse this wrapper's validated namespace.
+    module.parse_args = lambda: args
+    sys.argv = [
+        "run_frontier_prefix_replay.py",
+        "--trace", str(args.trace),
+        "--output-root", str(args.output_root),
+        "--config", args.config,
+        "--label", args.label,
+        "--max-tokens", str(args.max_tokens),
+    ]
+    if args.duration_s:
+        sys.argv += ["--duration-s", str(args.duration_s)]
+    if args.cache_root:
+        sys.argv += ["--cache-root", str(args.cache_root)]
+    module.main()
+
+    manifest_path = args.output_root / "manifest.json"
+    manifest = json.loads(manifest_path.read_text())
+    manifest["attention_profile_override"] = str(profile)
+    manifest["attention_profile_sha256"] = module.sha256(profile)
+    manifest["reuse_model_cache"] = args.reuse_model_cache
+    manifest["schema"] = "frontier-prefill-kvgrowth-replay-v1"
+    manifest_path.write_text(json.dumps(manifest, indent=2))
+    print(f"replay done: {args.output_root}")
+
+
+if __name__ == "__main__":
+    main()

runs/frontier-qwen30-vllm020-profile-v1/profile_vllm020_flashattn.py

@@ -37,6 +37,7 @@ def parse_args() -> argparse.Namespace:
     )
     parser.add_argument("--warmup-iters", type=int, default=3)
     parser.add_argument("--repeats", type=int, default=5)
+    parser.add_argument("--max-model-len", type=int, default=40960)
     parser.add_argument("--device", default="cuda:0")
     parser.add_argument("--profile-kv-update", action="store_true")
     parser.add_argument(
@@ -115,7 +116,7 @@ def main() -> None:
             trust_remote_code=False,
             dtype="bfloat16",
             seed=0,
-            max_model_len=40960,
+            max_model_len=args.max_model_len,
         )
         cache_config = CacheConfig(block_size=config.block_size, cache_dtype="auto")
         cache_config.num_gpu_blocks = max_num_blocks
@@ -124,7 +125,7 @@ def main() -> None:
         scheduler_config = SchedulerConfig(
             max_num_seqs=256,
             max_num_batched_tokens=8192,
-            max_model_len=40960,
+            max_model_len=args.max_model_len,
             is_encoder_decoder=False,
             enable_chunked_prefill=True,
         )

runs/frontier-s3-real-v0/qwen30_exact_trace_client.py

@@ -0,0 +1,268 @@
+#!/usr/bin/env python3
+"""Replay a private exact-trace anchor without emitting prompt text."""
+
+from __future__ import annotations
+
+import argparse
+import asyncio
+import hashlib
+import json
+import math
+from pathlib import Path
+from typing import Any
+
+
+TARGET_PASS_RATE = 0.95
+SSE_READ_BUFFER_BYTES = 8 * 1024 * 1024
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--host", default="127.0.0.1")
+    parser.add_argument("--port", type=int, required=True)
+    parser.add_argument("--requests-file", type=Path, required=True)
+    parser.add_argument("--output", type=Path, required=True)
+    parser.add_argument(
+        "--served-model",
+        default=None,
+        help="Optional server-side model alias used only for HTTP routing.",
+    )
+    parser.add_argument("--tpot-slo-ms", type=float, default=150.0)
+    parser.add_argument("--timeout-seconds", type=float, default=1800.0)
+    return parser.parse_args()
+
+
+def percentile(values: list[float], fraction: float) -> float | None:
+    if not values:
+        return None
+    ordered = sorted(values)
+    return ordered[math.ceil(fraction * len(ordered)) - 1]
+
+
+def ttft_slo_ms(input_tokens: int) -> float:
+    return 1000.0 + 1000.0 * input_tokens / 8000.0
+
+
+def row_vector_sha256(rows: list[dict[str, Any]]) -> str:
+    digest = hashlib.sha256()
+    for row in rows:
+        digest.update(
+            json.dumps(
+                [
+                    row["source_index"],
+                    row["arrived_at"],
+                    row["input_length"],
+                    row["output_length"],
+                    row["session_id"],
+                    row["runtime_block_ids"],
+                ],
+                separators=(",", ":"),
+            ).encode()
+        )
+        digest.update(b"\n")
+    return digest.hexdigest()
+
+
+async def request_one(
+    session: aiohttp.ClientSession,
+    row: dict[str, Any],
+    *,
+    scheduled_at: float,
+    benchmark_start: float,
+    tpot_slo_ms: float,
+    served_model: str | None,
+) -> dict[str, Any]:
+    loop = asyncio.get_running_loop()
+    delay = scheduled_at - loop.time()
+    if delay > 0:
+        await asyncio.sleep(delay)
+    admitted_at = loop.time()
+    record: dict[str, Any] = {
+        "source_index": int(row["source_index"]),
+        "session_id": int(row["session_id"]),
+        "scheduled_s": scheduled_at - benchmark_start,
+        "admitted_s": admitted_at - benchmark_start,
+        "admission_lag_ms": (admitted_at - scheduled_at) * 1000.0,
+        "input_tokens": int(row["input_length"]),
+        "requested_output_tokens": int(row["output_length"]),
+        "success": False,
+    }
+    body = dict(row["body"])
+    if served_model is not None:
+        body["model"] = served_model
+    body.update(
+        {
+            "temperature": 0,
+            "stream": True,
+            "stream_options": {"include_usage": True},
+            "return_token_ids": True,
+        }
+    )
+    try:
+        started = loop.time()
+        async with session.post("/v1/completions", json=body) as response:
+            if response.status != 200:
+                detail = (await response.text())[:1000]
+                raise RuntimeError(f"HTTP {response.status}: {detail}")
+            first_token_at = None
+            last_token_at = None
+            streamed_tokens = 0
+            usage = None
+            while True:
+                raw = await response.content.readline()
+                if not raw:
+                    break
+                line = raw.decode(errors="replace").strip()
+                if not line.startswith("data:"):
+                    continue
+                data = line[5:].strip()
+                if data == "[DONE]":
+                    break
+                payload = json.loads(data)
+                if payload.get("usage"):
+                    usage = payload["usage"]
+                emitted = 0
+                for choice in payload.get("choices") or []:
+                    token_ids = choice.get("token_ids") or []
+                    emitted += len(token_ids) if token_ids else int(bool(choice.get("text")))
+                if emitted:
+                    now = loop.time()
+                    first_token_at = first_token_at or now
+                    last_token_at = now
+                    streamed_tokens += emitted
+        finished = loop.time()
+        if first_token_at is None or last_token_at is None or usage is None:
+            raise RuntimeError("missing streaming token or usage")
+        actual_input = int(usage["prompt_tokens"])
+        actual_output = int(usage["completion_tokens"])
+        if actual_input != int(row["input_length"]) or actual_output != int(
+            row["output_length"]
+        ):
+            raise RuntimeError(f"usage mismatch: {actual_input}+{actual_output}")
+        ttft = (first_token_at - started) * 1000.0
+        tpot = (
+            (last_token_at - first_token_at) * 1000.0 / (actual_output - 1)
+            if actual_output > 1
+            else None
+        )
+        record.update(
+            {
+                "success": True,
+                "actual_input_tokens": actual_input,
+                "actual_output_tokens": actual_output,
+                "streamed_token_count": streamed_tokens,
+                "ttft_ms": ttft,
+                "tpot_ms": tpot,
+                "e2e_ms": (finished - started) * 1000.0,
+                "slo_pass": ttft <= ttft_slo_ms(actual_input)
+                and (tpot is None or tpot <= tpot_slo_ms),
+            }
+        )
+    except Exception as error:
+        record.update(
+            {
+                "error": f"{type(error).__name__}: {error}",
+                "slo_pass": False,
+            }
+        )
+    return record
+
+
+async def replay(args: argparse.Namespace, rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
+    import aiohttp
+
+    timeout = aiohttp.ClientTimeout(total=args.timeout_seconds)
+    connector = aiohttp.TCPConnector(limit=0, ttl_dns_cache=300, force_close=True)
+    benchmark_start = asyncio.get_running_loop().time() + 2.0
+    async with aiohttp.ClientSession(
+        base_url=f"http://{args.host}:{args.port}",
+        timeout=timeout,
+        connector=connector,
+        # return_token_ids can put the full long prompt token vector in one
+        # SSE event. aiohttp's readline limit is 2x read_bufsize; the default
+        # 512 KiB limit rejects valid 100k+ context responses.
+        read_bufsize=SSE_READ_BUFFER_BYTES,
+    ) as session:
+        tasks = [
+            asyncio.create_task(
+                request_one(
+                    session,
+                    row,
+                    scheduled_at=benchmark_start + float(row["arrived_at"]),
+                    benchmark_start=benchmark_start,
+                    tpot_slo_ms=args.tpot_slo_ms,
+                    served_model=args.served_model,
+                )
+            )
+            for row in rows
+        ]
+        return await asyncio.gather(*tasks)
+
+
+def main() -> None:
+    args = parse_args()
+    if args.tpot_slo_ms <= 0 or args.timeout_seconds <= 0:
+        raise ValueError("SLO and timeout must be positive")
+    rows = [json.loads(line) for line in args.requests_file.open() if line.strip()]
+    if not rows:
+        raise ValueError("requests file is empty")
+    arrivals = [float(row["arrived_at"]) for row in rows]
+    if any(right < left for left, right in zip(arrivals, arrivals[1:])):
+        raise ValueError("request arrival order drift")
+    requests = asyncio.run(replay(args, rows))
+    requests.sort(key=lambda row: int(row["source_index"]))
+    completed = [row for row in requests if row["success"]]
+    passed = sum(bool(row["slo_pass"]) for row in requests)
+    ttfts = [float(row["ttft_ms"]) for row in completed]
+    tpots = [
+        float(row["tpot_ms"])
+        for row in completed
+        if row["tpot_ms"] is not None
+    ]
+    pass_rate = passed / len(requests)
+    payload = {
+        "schema": "qwen30-exact-trace-anchor-v1",
+        "contract": {
+            "requests_file": str(args.requests_file.resolve()),
+            "requests_file_sha256": hashlib.sha256(
+                args.requests_file.read_bytes()
+            ).hexdigest(),
+            "row_vector_sha256": row_vector_sha256(rows),
+            "requests": len(rows),
+            "first_arrival_s": arrivals[0],
+            "last_arrival_s": arrivals[-1],
+            "arrival": "original_trace_timestamp_and_order",
+            "input_output_prompt": "exact_source_values",
+            "served_model_alias": args.served_model,
+            "http_connection_reuse": False,
+            "sse_read_buffer_bytes": SSE_READ_BUFFER_BYTES,
+            "ttft_slo": "1000ms + 1000ms * input_tokens / 8000",
+            "tpot_slo_ms": args.tpot_slo_ms,
+            "target_pass_rate": TARGET_PASS_RATE,
+            "prompt_text_emitted": False,
+        },
+        "summary": {
+            "completed": len(completed),
+            "failed": len(requests) - len(completed),
+            "passed": passed,
+            "pass_rate": pass_rate,
+            "feasible": pass_rate >= TARGET_PASS_RATE,
+            "ttft_p50_ms": percentile(ttfts, 0.50),
+            "ttft_p95_ms": percentile(ttfts, 0.95),
+            "tpot_p50_ms": percentile(tpots, 0.50),
+            "tpot_p95_ms": percentile(tpots, 0.95),
+            "admission_lag_p95_ms": percentile(
+                [float(row["admission_lag_ms"]) for row in requests], 0.95
+            ),
+        },
+        "requests": requests,
+    }
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
+    print(json.dumps(payload["summary"], sort_keys=True), flush=True)
+    if len(completed) != len(requests):
+        raise SystemExit(2)
+
+
+if __name__ == "__main__":
+    main()

runs/frontier-s3-real-v0/qwen30_prefill_client.py

@@ -0,0 +1,253 @@
+#!/usr/bin/env python3
+"""Open-loop fixed-shape workload for one real offered-load anchor."""
+
+from __future__ import annotations
+
+import argparse
+import concurrent.futures
+import hashlib
+import http.client
+import json
+import math
+import time
+from pathlib import Path
+from typing import Any
+
+
+TARGET_PASS_RATE = 0.95
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--host", default="127.0.0.1")
+    parser.add_argument("--port", type=int, required=True)
+    parser.add_argument("--served-model", required=True)
+    parser.add_argument("--model-path", type=Path, required=True)
+    parser.add_argument("--rate", type=float, required=True)
+    parser.add_argument("--requests", type=int, default=64)
+    parser.add_argument("--input-tokens", type=int, default=2048)
+    parser.add_argument("--output-tokens", type=int, default=1)
+    parser.add_argument("--ttft-slo-ms", type=float)
+    parser.add_argument("--tpot-slo-ms", type=float, default=150.0)
+    parser.add_argument("--timeout-seconds", type=float, default=900.0)
+    parser.add_argument("--output", type=Path, required=True)
+    return parser.parse_args()
+
+
+def percentile(values: list[float], fraction: float) -> float | None:
+    if not values:
+        return None
+    ordered = sorted(values)
+    index = min(len(ordered) - 1, max(0, math.ceil(fraction * len(ordered)) - 1))
+    return ordered[index]
+
+
+def ttft_slo_ms(input_tokens: int) -> float:
+    return 1000.0 + 1000.0 * input_tokens / 8000.0
+
+
+def run_request(
+    *,
+    request_index: int,
+    scheduled_at: float,
+    benchmark_start: float,
+    args: argparse.Namespace,
+    prompt_ids: list[int],
+) -> dict[str, Any]:
+    delay = scheduled_at - time.perf_counter()
+    if delay > 0:
+        time.sleep(delay)
+    admitted = time.perf_counter()
+    record: dict[str, Any] = {
+        "request_index": request_index,
+        "scheduled_s": scheduled_at - benchmark_start,
+        "admitted_s": admitted - benchmark_start,
+        "admission_lag_ms": (admitted - scheduled_at) * 1000.0,
+        "success": False,
+    }
+    connection = http.client.HTTPConnection(args.host, args.port, timeout=args.timeout_seconds)
+    body = {
+        "model": args.served_model,
+        "prompt": prompt_ids,
+        "min_tokens": args.output_tokens,
+        "max_tokens": args.output_tokens,
+        "ignore_eos": True,
+        "temperature": 0,
+        "stream": True,
+        "stream_options": {"include_usage": True},
+        "return_token_ids": True,
+    }
+    try:
+        started = time.perf_counter()
+        connection.request(
+            "POST",
+            "/v1/completions",
+            body=json.dumps(body, separators=(",", ":")).encode(),
+            headers={"Content-Type": "application/json"},
+        )
+        response = connection.getresponse()
+        if response.status != 200:
+            raise RuntimeError(
+                f"HTTP {response.status}: {response.read().decode(errors='replace')}"
+            )
+        first_token_at = None
+        last_token_at = None
+        streamed_tokens = 0
+        usage = None
+        while True:
+            raw = response.readline()
+            if not raw:
+                break
+            line = raw.decode(errors="replace").strip()
+            if not line.startswith("data:"):
+                continue
+            data = line[5:].strip()
+            if data == "[DONE]":
+                break
+            payload = json.loads(data)
+            if payload.get("usage"):
+                usage = payload["usage"]
+            emitted = 0
+            for choice in payload.get("choices") or []:
+                token_ids = choice.get("token_ids") or []
+                emitted += len(token_ids) if token_ids else int(bool(choice.get("text")))
+            if emitted:
+                now = time.perf_counter()
+                first_token_at = first_token_at or now
+                last_token_at = now
+                streamed_tokens += emitted
+        finished = time.perf_counter()
+        if first_token_at is None or usage is None:
+            raise RuntimeError("missing streaming token or usage")
+        prompt_tokens = int(usage["prompt_tokens"])
+        completion_tokens = int(usage["completion_tokens"])
+        if prompt_tokens != args.input_tokens or completion_tokens != args.output_tokens:
+            raise RuntimeError(f"usage mismatch: {prompt_tokens}+{completion_tokens}")
+        ttft = (first_token_at - started) * 1000.0
+        tpot = (
+            (last_token_at - first_token_at) * 1000.0 / (completion_tokens - 1)
+            if completion_tokens > 1
+            else None
+        )
+        record.update(
+            {
+                "success": True,
+                "prompt_tokens": prompt_tokens,
+                "completion_tokens": completion_tokens,
+                "streamed_token_count": streamed_tokens,
+                "ttft_ms": ttft,
+                "tpot_ms": tpot,
+                "e2e_ms": (finished - started) * 1000.0,
+                "slo_pass": ttft <= args.ttft_slo_ms
+                and (tpot is None or tpot <= args.tpot_slo_ms),
+            }
+        )
+    except Exception as error:  # Failed requests remain in the SLO denominator.
+        record["error"] = f"{type(error).__name__}: {error}"
+        record["slo_pass"] = False
+    finally:
+        connection.close()
+    return record
+
+
+def main() -> None:
+    args = parse_args()
+    if min(
+        args.rate,
+        args.requests,
+        args.input_tokens,
+        args.output_tokens,
+        args.tpot_slo_ms,
+    ) <= 0:
+        raise ValueError("rate, requests, tokens, and SLO must be positive")
+    if args.ttft_slo_ms is None:
+        args.ttft_slo_ms = ttft_slo_ms(args.input_tokens)
+    if args.ttft_slo_ms <= 0:
+        raise ValueError("TTFT SLO must be positive")
+    from transformers import AutoTokenizer
+
+    tokenizer = AutoTokenizer.from_pretrained(args.model_path, trust_remote_code=True)
+    excluded = set(tokenizer.all_special_ids)
+    candidates = [
+        token_id for token_id in range(tokenizer.vocab_size) if token_id not in excluded
+    ]
+    if len(candidates) < args.requests + 1:
+        raise RuntimeError("tokenizer has too few non-special token IDs")
+    base_id = candidates[0]
+    prompts = [
+        [candidates[index + 1], *([base_id] * (args.input_tokens - 1))]
+        for index in range(args.requests)
+    ]
+    prompt_hash = hashlib.sha256(
+        "\n".join(",".join(map(str, prompt)) for prompt in prompts).encode()
+    ).hexdigest()
+
+    benchmark_start = time.perf_counter() + 2.0
+    with concurrent.futures.ThreadPoolExecutor(max_workers=args.requests) as pool:
+        futures = [
+            pool.submit(
+                run_request,
+                request_index=index,
+                scheduled_at=benchmark_start + index / args.rate,
+                benchmark_start=benchmark_start,
+                args=args,
+                prompt_ids=prompts[index],
+            )
+            for index in range(args.requests)
+        ]
+        requests = [future.result() for future in futures]
+    requests.sort(key=lambda row: int(row["request_index"]))
+    completed = [row for row in requests if row["success"]]
+    passed = sum(bool(row["slo_pass"]) for row in requests)
+    ttfts = [float(row["ttft_ms"]) for row in completed]
+    tpots = [
+        float(row["tpot_ms"])
+        for row in completed
+        if row["tpot_ms"] is not None
+    ]
+    pass_rate = passed / len(requests)
+    payload = {
+        "schema": "qwen30-fixed-rate-anchor-v2",
+        "workload": {
+            "offered_request_rate": args.rate,
+            "request_count": args.requests,
+            "input_tokens": args.input_tokens,
+            "output_tokens": args.output_tokens,
+            "prefix_caching": False,
+            "arrival": "open_loop_uniform",
+            "last_scheduled_arrival_s": (args.requests - 1) / args.rate,
+            "prompt_vector_sha256": prompt_hash,
+        },
+        "summary": {
+            "completed": len(completed),
+            "failed": len(requests) - len(completed),
+            "ttft_p50_ms": percentile(ttfts, 0.50),
+            "ttft_p95_ms": percentile(ttfts, 0.95),
+            "ttft_max_ms": max(ttfts) if ttfts else None,
+            "tpot_p50_ms": percentile(tpots, 0.50),
+            "tpot_p95_ms": percentile(tpots, 0.95),
+            "tpot_max_ms": max(tpots) if tpots else None,
+            "admission_lag_max_ms": max(
+                float(row["admission_lag_ms"]) for row in requests
+            ),
+            "slo": {
+                "ttft_threshold_ms": args.ttft_slo_ms,
+                "tpot_threshold_ms": (
+                    args.tpot_slo_ms if args.output_tokens > 1 else None
+                ),
+                "passed": passed,
+                "pass_rate": pass_rate,
+                "feasible": pass_rate >= TARGET_PASS_RATE,
+            },
+        },
+        "requests": requests,
+    }
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
+    print(json.dumps(payload["summary"], sort_keys=True), flush=True)
+    if len(completed) != args.requests:
+        raise SystemExit(2)
+
+
+if __name__ == "__main__":
+    main()

runs/frontier-s3-real-v0/remap_hash_blocks.py

@@ -0,0 +1,515 @@
+#!/usr/bin/env python3
+"""Remap source hashes to real-token 16-token blocks and replay prompts."""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import hashlib
+import json
+import math
+from contextlib import ExitStack
+from itertools import zip_longest
+from pathlib import Path
+from typing import Any, Iterator
+
+from trace_utils import (
+    TARGET_BLOCK_SIZE,
+    expand_hash_ids,
+    input_length,
+    iter_jsonl,
+    output_length,
+    parse_hash_ids,
+    session_id,
+    sha256,
+    synthetic_tokens,
+    token_block_identity_records,
+    timestamp,
+    write_json,
+)
+
+
+CSV_FIELDS = (
+    "arrived_at",
+    "num_prefill_tokens",
+    "num_decode_tokens",
+    "session_id",
+    "block_hash_ids",
+)
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--input", type=Path, required=True)
+    parser.add_argument("--prompt", type=Path)
+    parser.add_argument("--tokenizer", type=Path)
+    parser.add_argument("--input-is-remapped", action="store_true")
+    parser.add_argument("--output-root", type=Path, required=True)
+    parser.add_argument("--rho", type=float, default=1.0)
+    parser.add_argument("--start-timestamp", type=float)
+    parser.add_argument("--duration-s", type=float)
+    parser.add_argument("--vocab-size", type=int, default=151936)
+    parser.add_argument("--token-offset", type=int, default=1024)
+    parser.add_argument("--served-model", default="qwen3-30b-s3-real")
+    parser.add_argument("--source-block-size", type=int, default=64)
+    parser.add_argument(
+        "--workload-mode",
+        choices=("prefill_decode", "prefill_only"),
+        default="prefill_decode",
+    )
+    parser.add_argument("--validate-parents", action="store_true")
+    parser.add_argument("--max-total-tokens", type=int)
+    parser.add_argument("--frontier-only", action="store_true")
+    return parser.parse_args()
+
+
+def keep_row(row: dict[str, Any], rho: float) -> bool:
+    if not 0 < rho <= 1:
+        raise ValueError("rho must be in (0, 1]")
+    if rho == 1:
+        return True
+    if "sampling_u" not in row:
+        raise ValueError("sampling_u is required for rho < 1")
+    return float(row["sampling_u"]) <= rho
+
+
+def row_identity(row: dict[str, Any], source_index: int) -> Any:
+    return row.get("chat_id", source_index)
+
+
+def iter_source_prompt_rows(
+    source: Path, prompt: Path | None, input_is_remapped: bool
+) -> Iterator[tuple[int, dict[str, Any], dict[str, Any] | None]]:
+    if input_is_remapped:
+        for source_index, row in enumerate(iter_jsonl(source)):
+            yield source_index, row, None
+        return
+    if prompt is None:
+        for source_index, row in enumerate(iter_jsonl(source)):
+            yield source_index, row, None
+        return
+    with source.open() as source_stream, prompt.open() as prompt_stream:
+        for source_index, pair in enumerate(zip_longest(source_stream, prompt_stream)):
+            source_line, prompt_line = pair
+            if source_line is None or prompt_line is None:
+                raise ValueError("trace/prompt window row-count mismatch")
+            if not source_line.strip() and not prompt_line.strip():
+                continue
+            if not source_line.strip() or not prompt_line.strip():
+                raise ValueError(f"trace/prompt blank-line mismatch at row {source_index + 1}")
+            row = json.loads(source_line)
+            prompt_row = json.loads(prompt_line)
+            if (
+                row.get("chat_id") != prompt_row.get("chat_id")
+                or row.get("turn") != prompt_row.get("turn")
+            ):
+                raise ValueError(f"trace/prompt alignment failed at row {source_index + 1}")
+            yield source_index, row, prompt_row
+
+
+def load_tokenizer(path: Path | None) -> Any:
+    if path is None:
+        raise ValueError("--tokenizer is required for real prompt tokenization")
+    from transformers import AutoTokenizer
+
+    return AutoTokenizer.from_pretrained(path, trust_remote_code=True, local_files_only=True)
+
+
+def prompt_token_ids(tokenizer: Any, prompt: str) -> list[int]:
+    encoded = tokenizer(
+        prompt,
+        add_special_tokens=False,
+        padding=False,
+        truncation=False,
+    )["input_ids"]
+    return [int(token) for token in encoded]
+
+
+def materialize(args: argparse.Namespace, *, tokenizer: Any | None = None) -> dict[str, Any]:
+    if args.output_root.exists():
+        raise ValueError(f"refusing to overwrite {args.output_root}")
+    args.output_root.mkdir(parents=True)
+    frontier_path = args.output_root / "frontier.csv"
+    real_path = args.output_root / "real_requests.jsonl"
+    selected_path = args.output_root / "selected-remapped.jsonl"
+    start = args.start_timestamp
+    end = start + args.duration_s if start is not None and args.duration_s is not None else None
+    first_selected_timestamp: float | None = None
+    request_count = 0
+    source_rows = 0
+    prefill_tokens = 0
+    decode_tokens = 0
+    target_blocks = 0
+    source_blocks = 0
+    parent_sequences: dict[Any, list[int]] = {}
+    parent_links_checked = 0
+    parent_links_outside = 0
+    parent_prefix_common_blocks = 0
+    parent_tail_nonreused_blocks = 0
+    excluded_over_max_tokens = 0
+    real_prompt_requests = 0
+    synthetic_fallback_requests = 0
+    input_length_mismatches = 0
+    source_hash_count_mismatches = 0
+    identity_collisions = 0
+    source_to_runtime_conflicts = 0
+    runtime_to_source_conflicts = 0
+    identity_to_witness: dict[int, bytes] = {}
+    source_to_runtime: dict[int, tuple[int, ...]] = {}
+    runtime_to_source: dict[tuple[int, ...], int] = {}
+    row_vector_digest = hashlib.sha256()
+    upstream_manifest_path: Path | None = None
+    upstream_block_contract: dict[str, Any] | None = None
+
+    source_block_size = int(getattr(args, "source_block_size", 64))
+    workload_mode = str(getattr(args, "workload_mode", "prefill_decode"))
+    if source_block_size <= 0 or source_block_size % TARGET_BLOCK_SIZE:
+        raise ValueError("source block size must be a positive multiple of 16")
+    subblocks_per_source = source_block_size // TARGET_BLOCK_SIZE
+    if not args.input_is_remapped and tokenizer is None and args.prompt is not None:
+        tokenizer = load_tokenizer(args.tokenizer)
+    if args.input_is_remapped:
+        upstream_manifest_path = args.input.parent / "manifest.json"
+        if not upstream_manifest_path.is_file():
+            raise FileNotFoundError(f"pre-remapped input requires sibling manifest: {upstream_manifest_path}")
+        upstream = json.loads(upstream_manifest_path.read_text())
+        source_block_size = int(upstream["source_block_size"])
+        subblocks_per_source = source_block_size // TARGET_BLOCK_SIZE
+        if int(upstream["target_block_size"]) != TARGET_BLOCK_SIZE:
+            raise ValueError("upstream remap target block size is not 16")
+        upstream_block_contract = dict(upstream["block_contract"])
+
+    with ExitStack() as stack:
+        frontier_stream = stack.enter_context(frontier_path.open("w", newline=""))
+        selected_stream = (
+            None
+            if args.input_is_remapped and args.frontier_only
+            else stack.enter_context(selected_path.open("w"))
+        )
+        real_stream = None if args.frontier_only else stack.enter_context(real_path.open("w"))
+        writer = csv.DictWriter(frontier_stream, fieldnames=CSV_FIELDS, lineterminator="\n")
+        writer.writeheader()
+        for source_index, row, prompt_row in iter_source_prompt_rows(
+            args.input, args.prompt, args.input_is_remapped
+        ):
+            source_rows += 1
+            ts = timestamp(row)
+            if start is not None and ts < start:
+                continue
+            if end is not None and ts >= end:
+                continue
+            if not keep_row(row, args.rho):
+                continue
+            isl = input_length(row)
+            osl = 1 if workload_mode == "prefill_only" else output_length(row)
+            if args.max_total_tokens is not None and isl + osl > args.max_total_tokens:
+                excluded_over_max_tokens += 1
+                continue
+            original_hashes = parse_hash_ids(row.get("hash_ids"))
+            if args.input_is_remapped:
+                block_ids = [int(value) for value in row["hash_ids_16"]]
+                if len(block_ids) != math.ceil(isl / TARGET_BLOCK_SIZE):
+                    raise ValueError(f"pre-remapped 16-block count mismatch for chat_id={row.get('chat_id')!r}")
+                if len(original_hashes) != math.ceil(isl / source_block_size):
+                    raise ValueError(
+                        f"pre-remapped {source_block_size}-block count mismatch "
+                        f"for chat_id={row.get('chat_id')!r}"
+                    )
+                request_prompt = row["prompt"]
+                prompt_kind = str(row["prompt_source"])
+                is_real_prompt = prompt_kind in ("real_text", "real_tokens")
+                real_prompt_requests += int(is_real_prompt)
+                synthetic_fallback_requests += int(not is_real_prompt)
+            else:
+                prompt_text = prompt_row.get("prompt") if prompt_row is not None else None
+                if isinstance(prompt_text, str) and prompt_text:
+                    assert tokenizer is not None
+                    tokens = prompt_token_ids(tokenizer, prompt_text)
+                    input_length_mismatches += int(len(tokens) != isl)
+                    if len(tokens) != isl:
+                        raise ValueError(
+                            f"prompt token length mismatch for chat_id={row.get('chat_id')!r}: "
+                            f"trace={isl}, tokenizer={len(tokens)}"
+                        )
+                    expected_source_blocks = math.ceil(len(tokens) / source_block_size)
+                    source_hash_count_mismatches += int(len(original_hashes) != expected_source_blocks)
+                    if len(original_hashes) != expected_source_blocks:
+                        raise ValueError(
+                            f"strict {source_block_size}-block contract failed "
+                            f"for chat_id={row.get('chat_id')!r}: "
+                            f"expected={expected_source_blocks}, got={len(original_hashes)}"
+                        )
+                    records = token_block_identity_records(tokens)
+                    block_ids = [identity for identity, _ in records]
+                    for runtime_id, witness in records:
+                        previous = identity_to_witness.setdefault(runtime_id, witness)
+                        identity_collisions += int(previous != witness)
+                    for block_index, source_hash in enumerate(original_hashes):
+                        begin = block_index * subblocks_per_source
+                        relation = tuple(
+                            block_ids[begin : begin + subblocks_per_source]
+                        )
+                        previous_relation = source_to_runtime.setdefault(source_hash, relation)
+                        source_to_runtime_conflicts += int(previous_relation != relation)
+                        previous_source = runtime_to_source.setdefault(relation, source_hash)
+                        runtime_to_source_conflicts += int(previous_source != source_hash)
+                    request_prompt = tokens
+                    prompt_kind = "real_tokens"
+                    real_prompt_requests += 1
+                else:
+                    content_block_ids = expand_hash_ids(
+                        original_hashes,
+                        isl,
+                        source_block_size=source_block_size,
+                    )
+                    request_prompt = synthetic_tokens(
+                        content_block_ids,
+                        isl,
+                        vocab_size=args.vocab_size,
+                        token_offset=args.token_offset,
+                    )
+                    records = token_block_identity_records(request_prompt)
+                    block_ids = [identity for identity, _ in records]
+                    for runtime_id, witness in records:
+                        previous = identity_to_witness.setdefault(runtime_id, witness)
+                        identity_collisions += int(previous != witness)
+                    prompt_kind = "synthetic_missing_prompt_fallback"
+                    synthetic_fallback_requests += 1
+            # vLLM only inserts complete physical blocks into the prefix cache.
+            # Keep the final partial block identity in selected-remapped.jsonl
+            # for exact token/block validation, but never expose it as a
+            # cacheable Frontier block. Otherwise an exact repeated prompt can
+            # report ceil(ISL / 16) cached blocks, exceeding ISL and leaving the
+            # request permanently unschedulable.
+            cacheable_block_ids = block_ids[: isl // TARGET_BLOCK_SIZE]
+            parent = row.get("parent_chat_id")
+            if args.validate_parents and parent not in (None, "", -1, "-1"):
+                parent_blocks = parent_sequences.get(parent)
+                if parent_blocks is None:
+                    parent_links_outside += 1
+                else:
+                    parent_links_checked += 1
+                    # Real multi-turn chats re-tokenize the full turn-1 history into
+                    # the turn-2 prompt. Because source blocks are not necessarily
+                    # aligned to turn boundaries, the parent's trailing partial block
+                    # does not survive as a child prefix: vLLM/Frontier reuse the
+                    # parent's complete leading blocks and recompute from the first
+                    # unaligned block. So we require a contiguous common prefix and
+                    # only tolerate a bounded mismatch confined to the parent tail.
+                    common = 0
+                    for a, b in zip(cacheable_block_ids, parent_blocks):
+                        if a != b:
+                            break
+                        common += 1
+                    # Tail slack is one possibly-partial source block plus one
+                    # runtime block for tokenizer boundary effects.
+                    max_parent_tail_slack = subblocks_per_source + 1
+                    if len(parent_blocks) - common > max_parent_tail_slack:
+                        raise ValueError(
+                            f"parent prefix violation beyond tail slack: "
+                            f"parent={parent!r}, child={row.get('chat_id')!r}, "
+                            f"common={common}, parent_blocks={len(parent_blocks)}"
+                        )
+                    parent_prefix_common_blocks += common
+                    parent_tail_nonreused_blocks += len(parent_blocks) - common
+            parent_sequences[row_identity(row, source_index)] = cacheable_block_ids
+
+            if first_selected_timestamp is None:
+                first_selected_timestamp = ts
+            arrival_origin = start if start is not None else first_selected_timestamp
+            arrived_at = ts - arrival_origin
+            sid = session_id(row)
+            writer.writerow(
+                {
+                    "arrived_at": f"{arrived_at:.9f}",
+                    "num_prefill_tokens": isl,
+                    "num_decode_tokens": osl,
+                    "session_id": sid,
+                    # "[]" is the explicit no-cacheable-block encoding accepted
+                    # by Frontier. An empty CSV field is treated as missing
+                    # metadata and rejected before scheduling.
+                    "block_hash_ids": (
+                        "|".join(str(value) for value in cacheable_block_ids)
+                        if cacheable_block_ids
+                        else "[]"
+                    ),
+                }
+            )
+            row_vector_digest.update(
+                json.dumps(
+                    [
+                        row.get("source_index", source_index),
+                        arrived_at,
+                        isl,
+                        osl,
+                        sid,
+                        cacheable_block_ids,
+                    ],
+                    separators=(",", ":"),
+                ).encode()
+            )
+            row_vector_digest.update(b"\n")
+            real_request = {
+                "source_index": row.get("source_index", source_index),
+                "chat_id": row.get("chat_id"),
+                "parent_chat_id": parent,
+                "arrived_at": arrived_at,
+                "input_length": isl,
+                "output_length": osl,
+                "session_id": sid,
+                "runtime_block_ids": cacheable_block_ids,
+                "body": {
+                    "model": args.served_model,
+                    "prompt": request_prompt,
+                    "min_tokens": osl,
+                    "max_tokens": osl,
+                    "ignore_eos": True,
+                    "stream": True,
+                },
+            }
+            if real_stream is not None:
+                real_stream.write(json.dumps(real_request, separators=(",", ":")) + "\n")
+            if selected_stream is not None:
+                selected_stream.write(
+                    json.dumps(
+                        {
+                            **row,
+                            "output_length": osl,
+                            "source_index": row.get("source_index", source_index),
+                            "arrived_at": arrived_at,
+                            "hash_ids_16": block_ids,
+                            "prompt": request_prompt,
+                            "prompt_source": prompt_kind,
+                        },
+                        separators=(",", ":"),
+                    )
+                    + "\n"
+                )
+            request_count += 1
+            prefill_tokens += isl
+            decode_tokens += osl
+            source_blocks += len(original_hashes)
+            target_blocks += len(cacheable_block_ids)
+
+    if request_count == 0:
+        raise ValueError("rho/window selected no requests")
+    contract_failures = {
+        "input_length_mismatches": input_length_mismatches,
+        "source_hash_count_mismatches": source_hash_count_mismatches,
+        "runtime_identity_collisions": identity_collisions,
+        "source_to_runtime_relation_conflicts": source_to_runtime_conflicts,
+        "runtime_to_source_relation_conflicts": runtime_to_source_conflicts,
+    }
+    if upstream_block_contract is not None:
+        inherited_failures = {
+            key: value
+            for key, value in upstream_block_contract.items()
+            if key.endswith("mismatches") or key.endswith("collisions") or key.endswith("conflicts")
+        }
+        if any(inherited_failures.values()):
+            raise ValueError(f"upstream block contract failed: {inherited_failures}")
+        contract_failures = inherited_failures
+    if any(contract_failures.values()):
+        raise ValueError(f"source-to-16 real-token block contract failed: {contract_failures}")
+    manifest = {
+        "schema": "frontier-s3-real-remap-v2",
+        "source": str(args.input.resolve()),
+        "source_sha256": sha256(args.input),
+        "prompt_window": str(args.prompt.resolve()) if args.prompt else None,
+        "prompt_window_sha256": sha256(args.prompt) if args.prompt else None,
+        "upstream_remap_manifest": str(upstream_manifest_path.resolve()) if upstream_manifest_path else None,
+        "upstream_remap_manifest_sha256": sha256(upstream_manifest_path) if upstream_manifest_path else None,
+        "rho": args.rho,
+        "sampling_rule": "keep iff sampling_u <= rho (SimFid convention; rho=1 keeps all rows)",
+        "window": {"start_timestamp": start, "duration_s": args.duration_s},
+        "requests": request_count,
+        "source_rows_scanned": source_rows,
+        "excluded_over_max_total_tokens": excluded_over_max_tokens,
+        "max_total_tokens_filter": args.max_total_tokens,
+        "total_prefill_tokens": prefill_tokens,
+        "total_decode_tokens": decode_tokens,
+        "source_blocks": source_blocks,
+        "target_16_blocks": target_blocks,
+        "paired_row_vector_sha256": row_vector_digest.hexdigest(),
+        "source_block_size": source_block_size,
+        "target_block_size": TARGET_BLOCK_SIZE,
+        "prefix_cache_blocks": "complete 16-token blocks only; final partial block excluded",
+        "workload_mode": workload_mode,
+        "mapping": (
+            "real prompt tokens: BLAKE2b-128(parent runtime identity, exact 16-token block); "
+            "missing prompt fallback: source hashes first define deterministic 16-token "
+            "content blocks, then the same parent-sensitive BLAKE2b-128 runtime identity "
+            "contract is applied"
+        ),
+        "prompt_contract": {
+            "real_prompt_requests": real_prompt_requests,
+            "synthetic_fallback_requests": synthetic_fallback_requests,
+            "input_is_pre_remapped": args.input_is_remapped,
+            "tokenizer": str(args.tokenizer.resolve()) if args.tokenizer else None,
+        },
+        "block_contract": {
+            **contract_failures,
+            "unique_runtime_identities": (
+                upstream_block_contract.get("unique_runtime_identities", 0)
+                if upstream_block_contract is not None
+                else len(identity_to_witness)
+            ),
+            "unique_source_hash_relations": (
+                upstream_block_contract.get("unique_source_hash_relations", 0)
+                if upstream_block_contract is not None
+                else len(source_to_runtime)
+            ),
+            "unique_runtime_relations": (
+                upstream_block_contract.get("unique_runtime_relations", 0)
+                if upstream_block_contract is not None
+                else len(runtime_to_source)
+            ),
+            "source_runtime_granularity": (
+                f"one {source_block_size}-token source hash to "
+                f"{subblocks_per_source} consecutive 16-token identities; "
+                "final partial source block may have fewer"
+            ),
+        },
+        "synthetic_fallback_contract": {
+            "vocab_size": args.vocab_size,
+            "token_offset": args.token_offset,
+            "encoding": "injective base-(vocab_size-token_offset), 16 little-endian digits",
+            "same_16_block_hash_same_tokens": True,
+            "different_16_block_hash_different_tokens": True,
+            "runtime_identity_is_parent_sensitive": True,
+        },
+        "parent_validation": {
+            "enabled": args.validate_parents,
+            "links_checked": parent_links_checked,
+            "links_outside_selected_stream": parent_links_outside,
+            "rule": (
+                "contiguous common prefix required; mismatch tolerated only "
+                f"within parent trailing partial {source_block_size}-token block "
+                f"(<= {subblocks_per_source + 1} runtime blocks)"
+            ),
+            "parent_prefix_common_blocks": parent_prefix_common_blocks,
+            "parent_tail_nonreused_blocks": parent_tail_nonreused_blocks,
+            "mean_tail_nonreused_per_link": (
+                parent_tail_nonreused_blocks / parent_links_checked
+                if parent_links_checked
+                else 0.0
+            ),
+        },
+        "frontier_csv": str(frontier_path.resolve()),
+        "frontier_csv_sha256": sha256(frontier_path),
+        "real_requests": str(real_path.resolve()) if real_path.is_file() else None,
+        "real_requests_sha256": sha256(real_path) if real_path.is_file() else None,
+        "selected_remapped": str(selected_path.resolve()) if selected_path.is_file() else None,
+        "selected_remapped_sha256": sha256(selected_path) if selected_path.is_file() else None,
+    }
+    write_json(args.output_root / "manifest.json", manifest)
+    return manifest
+
+
+def main() -> None:
+    args = parse_args()
+    print(json.dumps(materialize(args), sort_keys=True))
+
+
+if __name__ == "__main__":
+    main()

runs/frontier-s3-real-v0/run_frontier_prefix_replay.py

@@ -0,0 +1,235 @@
+#!/usr/bin/env python3
+"""Replay a remapped trace with S1 joint injection and prefix caching enabled."""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import os
+import subprocess
+from collections import Counter
+from pathlib import Path
+from typing import Any
+
+from trace_utils import distribution, sha256, write_json
+
+
+ROOT = Path(__file__).resolve().parent
+REPO = ROOT.parents[1]
+REFERENCE = REPO / "runs/frontier-collective-joint-v0/counterfactual/joint-r2/manifest.json"
+EXPECTED_FRONTIER_COMMIT = "deadc4a321f0baaa534c6ebd17f974123733cdc2"
+MAX_CURVE_BATCH = 32
+
+
+csv.field_size_limit(16 * 1024 * 1024)
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--trace", type=Path, required=True)
+    parser.add_argument("--output-root", type=Path, required=True)
+    parser.add_argument("--config", choices=("tp4_mns16", "tp2_mns16"), default="tp4_mns16")
+    parser.add_argument("--label", required=True)
+    parser.add_argument("--max-tokens", type=int, required=True)
+    parser.add_argument("--duration-s", type=float)
+    parser.add_argument("--cache-root", type=Path)
+    return parser.parse_args()
+
+
+def replace_flag(argv: list[str], flag: str, value: str) -> None:
+    try:
+        index = argv.index(flag)
+    except ValueError as error:
+        raise ValueError(f"template command is missing {flag}") from error
+    argv[index + 1] = value
+
+
+def prepare_curves(reference: dict[str, Any], output_root: Path) -> dict[str, Any]:
+    inputs = output_root / "inputs"
+    inputs.mkdir(parents=True)
+    outputs: dict[str, Any] = {}
+    for kind, reference_key, filename in (
+        ("collective", "collective_curve", "collective-curve-b4-extrapolated.json"),
+        ("moe", "moe_curve", "fused-moe-curve-b4-extrapolated.json"),
+    ):
+        payload = json.loads(Path(reference[reference_key]).read_text())
+        curves = (
+            payload["curve_variants_ms_per_step"].values()
+            if kind == "collective" and "curve_variants_ms_per_step" in payload
+            else (payload,)
+        )
+        for curve in curves:
+            for points in curve.values():
+                b4 = points["4"]
+                for batch in range(5, MAX_CURVE_BATCH + 1):
+                    points[str(batch)] = b4
+        destination = inputs / filename
+        write_json(destination, payload)
+        outputs[kind] = str(destination)
+        outputs[f"{kind}_sha256"] = sha256(destination)
+    return outputs
+
+
+def find_one(root: Path, name: str) -> Path:
+    matches = list(root.glob(f"**/{name}"))
+    if len(matches) != 1:
+        raise ValueError(f"expected one {name} below {root}, found {matches}")
+    return matches[0]
+
+
+def read_csv(path: Path) -> list[dict[str, str]]:
+    with path.open(newline="") as stream:
+        return list(csv.DictReader(stream))
+
+
+def summarize(trace: Path, metrics_root: Path, duration_s: float) -> dict[str, Any]:
+    trace_rows = read_csv(trace)
+    metrics_path = find_one(metrics_root, "request_metrics.csv")
+    metric_rows = read_csv(metrics_path)
+    if len(trace_rows) != len(metric_rows):
+        raise ValueError(f"request count mismatch: trace={len(trace_rows)}, metrics={len(metric_rows)}")
+    system_path = find_one(metrics_root, "system_metrics.json")
+    system = json.loads(system_path.read_text())
+    ledger_path = find_one(metrics_root, "frontier_stage_batch_ledger.jsonl")
+    waiting_ms = [float(row["request_waiting_time_total"]) for row in metric_rows]
+    ttft_ms = [float(row["ttft"]) for row in metric_rows]
+    # TPOT is undefined for one-token outputs because there is no inter-token
+    # interval. Frontier records those cells as an empty CSV value.
+    tpot_ms = [
+        float(row["tpot"])
+        for row in metric_rows
+        if row.get("tpot") is not None and row["tpot"].strip()
+    ]
+    e2e_ms = [float(row["request_e2e_time"]) for row in metric_rows]
+    cached_tokens = [int(float(row.get("request_cached_prefill_tokens", 0))) for row in metric_rows]
+    query_blocks = [int(float(row.get("request_prefix_cache_query_blocks", 0))) for row in metric_rows]
+    hit_blocks = [int(float(row.get("request_prefix_cache_hit_blocks", 0))) for row in metric_rows]
+    arrivals = [float(row["arrived_at"]) for row in trace_rows]
+    prefill = [int(row["num_prefill_tokens"]) for row in trace_rows]
+    decode = [int(row["num_decode_tokens"]) for row in trace_rows]
+    completion_times = [arrival + e2e / 1000 for arrival, e2e in zip(arrivals, e2e_ms)]
+    batch_hist: Counter[int] = Counter()
+    with ledger_path.open() as stream:
+        for line in stream:
+            if not line.strip():
+                continue
+            row = json.loads(line)
+            tokens = row.get("request_num_tokens") or []
+            if tokens and all(int(token) == 1 for token in tokens):
+                batch_hist[len(tokens)] += 1
+    stages = sum(batch_hist.values())
+    prefix = system.get("prefix_cache_statistics") or {
+        "block_size_tokens": 16,
+        "requests": len(metric_rows),
+        "total_cached_prefill_tokens": sum(cached_tokens),
+        "total_query_blocks": sum(query_blocks),
+        "total_hit_blocks": sum(hit_blocks),
+        "hit_ratio": sum(hit_blocks) / sum(query_blocks) if sum(query_blocks) else 0.0,
+    }
+    return {
+        "schema": "frontier-s3-real-prefix-replay-summary-v1",
+        "requests": len(trace_rows),
+        "duration_s": duration_s,
+        "offered_load": {
+            "requests_per_s": len(trace_rows) / duration_s,
+            "prefill_tokens_per_s_raw": sum(prefill) / duration_s,
+            "prefill_tokens_per_s_after_prefix": (sum(prefill) - sum(cached_tokens)) / duration_s,
+            "decode_tokens_per_s": sum(decode) / duration_s,
+        },
+        "latency_ms": {
+            "waiting": distribution(waiting_ms),
+            "ttft": distribution(ttft_ms),
+            "tpot": distribution(tpot_ms),
+            "e2e": distribution(e2e_ms),
+        },
+        "drain": {
+            "last_arrival_s": max(arrivals),
+            "last_completion_s": max(completion_times),
+            "tail_after_last_arrival_s": max(completion_times) - max(arrivals),
+        },
+        "decode_batch": {
+            "stages": stages,
+            "histogram": dict(sorted(batch_hist.items())),
+            "share_gt_1": sum(count for batch, count in batch_hist.items() if batch > 1) / stages if stages else 0.0,
+            "share_gt_4": sum(count for batch, count in batch_hist.items() if batch > 4) / stages if stages else 0.0,
+            "max": max(batch_hist, default=0),
+        },
+        "prefix_cache": prefix,
+        "artifacts": {
+            "request_metrics": str(metrics_path),
+            "system_metrics": str(system_path),
+            "stage_ledger": str(ledger_path),
+        },
+    }
+
+
+def main() -> None:
+    args = parse_args()
+    args.trace = args.trace.resolve()
+    args.output_root = args.output_root.resolve()
+    args.cache_root = (args.cache_root or args.output_root.parent / "cache").resolve()
+    if args.output_root.exists():
+        raise ValueError(f"refusing to overwrite {args.output_root}")
+    reference = json.loads(REFERENCE.read_text())
+    frontier = Path(reference["frontier_checkout"])
+    commit = subprocess.check_output(["git", "-C", str(frontier), "rev-parse", "HEAD"], text=True).strip()
+    if commit != EXPECTED_FRONTIER_COMMIT or commit != reference["frontier_commit"]:
+        raise ValueError(f"Frontier commit drift: {commit}")
+    if subprocess.check_output(["git", "-C", str(frontier), "status", "--porcelain"], text=True).strip():
+        raise ValueError("Frontier checkout must be clean")
+    args.output_root.mkdir(parents=True)
+    curves = prepare_curves(reference, args.output_root)
+    cell = reference["cells"][args.config]
+    argv = list(cell["argv"])
+    replace_flag(argv, "--trace_request_generator_config_trace_file", str(args.trace))
+    replace_flag(argv, "--trace_request_generator_config_max_tokens", str(args.max_tokens))
+    replace_flag(argv, "--random_forrest_execution_time_predictor_config_prediction_max_tokens_per_request", str(args.max_tokens))
+    replace_flag(argv, "--metrics_config_output_dir", str(args.output_root / "metrics"))
+    replace_flag(argv, "--metrics_config_run_id", args.label)
+    replace_flag(argv, "--metrics_config_cache_dir", str(args.cache_root / "model"))
+    replace_flag(argv, "--vidur_cc_backend_config_cache_dir", str(args.cache_root / "cc"))
+    argv.extend(("--vllm_v1_scheduler_config_enable_prefix_caching", "--log_level", "warning", "--cluster_event_log_level", "WARNING"))
+    usage = args.output_root / "usage.json"
+    env = os.environ.copy()
+    env.update(
+        {
+            "CUDA_VISIBLE_DEVICES": "",
+            "PYTHONDONTWRITEBYTECODE": "1",
+            "PYTHONPATH": os.pathsep.join([str(frontier), *reference["python_dependency_roots"]]),
+            "FRONTIER_COLLECTIVE_CURVE": curves["collective"],
+            "FRONTIER_COLLECTIVE_CURVE_VARIANT": reference["collective_curve_variant"],
+            "FRONTIER_FUSED_MOE_CURVE": curves["moe"],
+            "FRONTIER_CURVE_USAGE": str(usage),
+        }
+    )
+    manifest = {
+        "schema": "frontier-s3-real-prefix-replay-v1",
+        "frontier_commit": commit,
+        "reference_manifest": str(REFERENCE.resolve()),
+        "reference_manifest_sha256": sha256(REFERENCE),
+        "trace": str(args.trace),
+        "trace_sha256": sha256(args.trace),
+        "config": args.config,
+        "prefix_caching": True,
+        "block_size": 16,
+        "max_tokens": args.max_tokens,
+        "argv": argv,
+        "curves": curves,
+    }
+    write_json(args.output_root / "manifest.json", manifest)
+    log_path = args.output_root / "sim.log"
+    with log_path.open("w") as log:
+        completed = subprocess.run(argv, cwd=frontier, env=env, stdout=log, stderr=subprocess.STDOUT, check=False)
+    if completed.returncode:
+        raise SystemExit(f"sim failed with exit code {completed.returncode}; see {log_path}")
+    with args.trace.open(newline="") as stream:
+        trace_rows = list(csv.DictReader(stream))
+    duration_s = args.duration_s or max(float(row["arrived_at"]) for row in trace_rows) or 1.0
+    summary = summarize(args.trace, args.output_root / "metrics", duration_s)
+    write_json(args.output_root / "summary.json", summary)
+    print(json.dumps(summary, sort_keys=True))
+
+
+if __name__ == "__main__":
+    main()

runs/frontier-s3-real-v0/run_full_real.sh

@@ -0,0 +1,227 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+RHO="${1:?usage: run_full_real.sh RHO CONFIG TRIAL PORT}"
+CONFIG="${2:?usage: run_full_real.sh RHO CONFIG TRIAL PORT}"
+TRIAL="${3:?usage: run_full_real.sh RHO CONFIG TRIAL PORT}"
+PORT="${4:?usage: run_full_real.sh RHO CONFIG TRIAL PORT}"
+CONTROL_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+VENV_ROOT="${VENV_ROOT:-/home/admin/cpfs/wjh/venvs/vllm-0.20.0-cu129-workload-regime-v2}"
+MODEL_ROOT="${MODEL_ROOT:-/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B}"
+SERVED_MODEL="qwen3-30b-s3-real-full"
+MAX_MODEL_LEN="${MAX_MODEL_LEN:-40960}"
+ALLOW_SYNTHETIC_PROMPTS="${ALLOW_SYNTHETIC_PROMPTS:-false}"
+ALLOW_LONG_CONTEXT_SERVER="${ALLOW_LONG_CONTEXT_SERVER:-false}"
+FLASHINFER_WORKSPACE_BASE="${FLASHINFER_WORKSPACE_BASE:-/tmp/wjh/flashinfer-frontier-s3-real-v0-${CONFIG}}"
+SERVER_PID=""
+TELEMETRY_PID=""
+
+[[ "${RHO}" =~ ^[A-Za-z0-9._-]+$ ]] || {
+  echo "ERROR: rho label contains unsupported characters: ${RHO}" >&2; exit 1;
+}
+if [[ -n "${TRACE_INPUT_ROOT:-}" ]]; then
+  INPUT_NAME=""
+else
+  case "${RHO}" in
+    0p00125) INPUT_NAME="canary-r0p00125" ;;
+    0p0025) INPUT_NAME="canary-r0p0025" ;;
+    0p005) INPUT_NAME="full-r0p005" ;;
+    0p01) INPUT_NAME="full-r0p01" ;;
+    0p02) INPUT_NAME="full-r0p02" ;;
+    *) echo "ERROR: unsupported built-in rho ${RHO}" >&2; exit 1 ;;
+  esac
+fi
+# Generic tp<N>_mns<M> parser: TP = tensor-parallel size, MNS = max-num-seqs.
+if [[ "${CONFIG}" =~ ^tp([0-9]+)_mns([0-9]+)$ ]]; then
+  TP="${BASH_REMATCH[1]}"; MNS="${BASH_REMATCH[2]}"
+else
+  echo "ERROR: unsupported config ${CONFIG}" >&2; exit 1
+fi
+[[ "${TRIAL}" == "1" || "${TRIAL}" == "2" ]] || {
+  echo "ERROR: trial must be 1 or 2" >&2; exit 1;
+}
+if (( MAX_MODEL_LEN > 40960 )) && [[ "${ALLOW_LONG_CONTEXT_SERVER}" != "true" ]]; then
+  echo "ERROR: MAX_MODEL_LEN>40960 requires ALLOW_LONG_CONTEXT_SERVER=true" >&2
+  exit 1
+fi
+if (( MAX_MODEL_LEN > 40960 )); then
+  export VLLM_CACHE_ROOT="${VLLM_CACHE_ROOT:-/tmp/wjh/vllm-cache-frontier-s3-real-longctx}"
+fi
+
+TRACE_INPUT_ROOT="${TRACE_INPUT_ROOT:-${CONTROL_ROOT}/inputs/${INPUT_NAME}}"
+REQUESTS_FILE="${TRACE_INPUT_ROOT}/real_requests.jsonl"
+INPUT_MANIFEST="${TRACE_INPUT_ROOT}/manifest.json"
+OUTPUT_ROOT="${OUTPUT_ROOT:-${CONTROL_ROOT}/outputs/full-real/rho-${RHO}/${CONFIG}/trial-${TRIAL}}"
+
+for path in "${VENV_ROOT}/bin/python" "${VENV_ROOT}/bin/vllm" \
+  "${MODEL_ROOT}/config.json" "${REQUESTS_FILE}" "${INPUT_MANIFEST}" \
+  "${CONTROL_ROOT}/qwen30_exact_trace_client.py" \
+  "${CONTROL_ROOT}/qwen30_prefill_client.py"; do
+  [[ -e "${path}" ]] || { echo "ERROR: missing ${path}" >&2; exit 1; }
+done
+[[ ! -e "${OUTPUT_ROOT}" ]] || { echo "ERROR: refusing to overwrite ${OUTPUT_ROOT}" >&2; exit 1; }
+mkdir -p "${OUTPUT_ROOT}/logs" "${OUTPUT_ROOT}/provenance" \
+  "${OUTPUT_ROOT}/results" "${OUTPUT_ROOT}/metrics" \
+  "${OUTPUT_ROOT}/telemetry" "${FLASHINFER_WORKSPACE_BASE}"
+exec > >(tee -a "${OUTPUT_ROOT}/logs/controller.log") 2>&1
+
+cleanup() {
+  if [[ -n "${TELEMETRY_PID}" ]] && kill -0 "${TELEMETRY_PID}" 2>/dev/null; then
+    kill "${TELEMETRY_PID}" 2>/dev/null || true
+  fi
+  TELEMETRY_PID=""
+  if [[ -n "${SERVER_PID}" ]] && kill -0 "${SERVER_PID}" 2>/dev/null; then
+    kill -TERM -- "-${SERVER_PID}" 2>/dev/null || true
+    for _ in $(seq 1 30); do
+      kill -0 "${SERVER_PID}" 2>/dev/null || break
+      sleep 1
+    done
+    kill -KILL -- "-${SERVER_PID}" 2>/dev/null || true
+  fi
+  SERVER_PID=""
+}
+trap cleanup EXIT INT TERM
+
+IFS=',' read -r -a GPU_IDS <<< "${CUDA_VISIBLE_DEVICES:?fleet GPU allocation is required}"
+[[ "${#GPU_IDS[@]}" -eq "${TP}" ]] || {
+  echo "ERROR: ${CONFIG} requires exactly ${TP} GPUs" >&2; exit 1;
+}
+nvidia-smi --query-gpu=index,name,memory.total,memory.used,utilization.gpu \
+  --format=csv,noheader,nounits > "${OUTPUT_ROOT}/provenance/gpus.before.csv"
+[[ "$(wc -l < "${OUTPUT_ROOT}/provenance/gpus.before.csv")" -eq 8 ]] || {
+  echo "ERROR: expected eight GPUs on host" >&2; exit 1;
+}
+grep -vq 'NVIDIA H20' "${OUTPUT_ROOT}/provenance/gpus.before.csv" && {
+  echo "ERROR: expected eight NVIDIA H20 GPUs" >&2; exit 1;
+}
+nvidia-smi --query-compute-apps=pid,process_name,used_memory --format=csv,noheader,nounits \
+  > "${OUTPUT_ROOT}/provenance/compute-apps.before.csv" || true
+if grep -Eq '^[[:space:]]*[0-9]+' "${OUTPUT_ROOT}/provenance/compute-apps.before.csv"; then
+  echo "ERROR: host is no longer fully idle" >&2
+  exit 1
+fi
+
+"${VENV_ROOT}/bin/python" - <<'PY' > "${OUTPUT_ROOT}/provenance/runtime.versions"
+import torch, transformers, vllm
+print(f"vllm={vllm.__version__}")
+print(f"torch={torch.__version__}")
+print(f"torch_cuda={torch.version.cuda}")
+print(f"transformers={transformers.__version__}")
+if vllm.__version__ != "0.20.0" or torch.version.cuda != "12.9":
+    raise SystemExit("runtime version mismatch")
+PY
+env | sort > "${OUTPUT_ROOT}/provenance/environment.txt"
+ps -eo user,pid,ppid,etimes,pcpu,pmem,rss,args --sort=pid \
+  > "${OUTPUT_ROOT}/provenance/processes.before.txt"
+sha256sum "${REQUESTS_FILE}" "${INPUT_MANIFEST}" \
+  "${CONTROL_ROOT}/qwen30_exact_trace_client.py" \
+  "${CONTROL_ROOT}/qwen30_prefill_client.py" "${BASH_SOURCE[0]}" \
+  > "${OUTPUT_ROOT}/provenance/sources.sha256"
+INPUT_MANIFEST="${INPUT_MANIFEST}" REQUESTS_FILE="${REQUESTS_FILE}" \
+  MAX_MODEL_LEN="${MAX_MODEL_LEN}" \
+  ALLOW_SYNTHETIC_PROMPTS="${ALLOW_SYNTHETIC_PROMPTS}" \
+  "${VENV_ROOT}/bin/python" - <<'PY'
+import json
+import os
+from pathlib import Path
+
+manifest = json.loads(Path(os.environ["INPUT_MANIFEST"]).read_text())
+source_block_size = int(manifest["source_block_size"])
+if source_block_size <= 0 or source_block_size % 16 or manifest["target_block_size"] != 16:
+    raise SystemExit("source-to-16 block-size contract failed")
+failures = {
+    key: value for key, value in manifest["block_contract"].items()
+    if key.endswith(("mismatches", "collisions", "conflicts"))
+}
+if any(failures.values()):
+    raise SystemExit(f"block/hash contract failed: {failures}")
+if (
+    manifest["prompt_contract"]["synthetic_fallback_requests"]
+    and os.environ["ALLOW_SYNTHETIC_PROMPTS"].lower() != "true"
+):
+    raise SystemExit("full real requires real prompt text for every request")
+with Path(os.environ["REQUESTS_FILE"]).open() as stream:
+    max_request_tokens = max(
+        int(row["input_length"]) + int(row["output_length"])
+        for row in map(json.loads, stream)
+    )
+if max_request_tokens > int(os.environ["MAX_MODEL_LEN"]):
+    raise SystemExit(
+        f"trace max request tokens {max_request_tokens} exceeds "
+        f"MAX_MODEL_LEN={os.environ['MAX_MODEL_LEN']}"
+    )
+PY
+
+export TOKENIZERS_PARALLELISM=false VLLM_USE_V1=1 TORCH_CUDA_ARCH_LIST=9.0 PREFIX_CACHING=true
+export HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 FLASHINFER_WORKSPACE_BASE
+HF_OVERRIDE_ARGS=()
+if [[ "${ALLOW_LONG_CONTEXT_SERVER}" == "true" ]]; then
+  export VLLM_ALLOW_LONG_MAX_MODEL_LEN=1
+  HF_OVERRIDE_ARGS=(--hf-overrides "{\"max_position_embeddings\":${MAX_MODEL_LEN}}")
+fi
+ulimit -n 65536
+setsid "${VENV_ROOT}/bin/vllm" serve "${MODEL_ROOT}" \
+  --host 127.0.0.1 --port "${PORT}" --served-model-name "${SERVED_MODEL}" \
+  --tensor-parallel-size "${TP}" --gpu-memory-utilization 0.92 \
+  "${HF_OVERRIDE_ARGS[@]}" \
+  --max-model-len "${MAX_MODEL_LEN}" --max-num-batched-tokens 8192 --max-num-seqs "${MNS}" \
+  --enable-prefix-caching --block-size 16 --enable-chunked-prefill --no-enable-log-requests \
+  --enable-logging-iteration-details \
+  > "${OUTPUT_ROOT}/logs/server.log" 2>&1 &
+SERVER_PID=$!
+READY=0
+for _ in $(seq 1 900); do
+  if curl -fsS --max-time 2 "http://127.0.0.1:${PORT}/v1/models" \
+    > "${OUTPUT_ROOT}/results/models.json" 2>/dev/null; then READY=1; break; fi
+  kill -0 "${SERVER_PID}" 2>/dev/null || { tail -200 "${OUTPUT_ROOT}/logs/server.log"; exit 1; }
+  sleep 3
+done
+[[ "${READY}" -eq 1 ]] || { echo "ERROR: server readiness timeout" >&2; exit 1; }
+
+"${VENV_ROOT}/bin/python" "${CONTROL_ROOT}/qwen30_prefill_client.py" \
+  --port "${PORT}" --served-model "${SERVED_MODEL}" --model-path "${MODEL_ROOT}" \
+  --rate 1 --requests 4 --input-tokens 512 --output-tokens 1 \
+  --output "${OUTPUT_ROOT}/results/warmup.json"
+curl -fsS "http://127.0.0.1:${PORT}/metrics" > "${OUTPUT_ROOT}/metrics/before.prom"
+(
+  while true; do
+    ts=$(date -u +%s.%N)
+    snap=$(curl -fsS "http://127.0.0.1:${PORT}/metrics" 2>/dev/null \
+      | grep -E 'vllm:(num_requests_running|num_requests_waiting|prefix_cache_hits|prefix_cache_queries|gpu_prefix_cache_hit_rate|gpu_cache_usage)' \
+      | tr '\n' '|')
+    printf '{"wall_time_epoch_s": %s, "metrics": "%s"}\n' "${ts}" "${snap}" \
+      >> "${OUTPUT_ROOT}/telemetry/server-state.jsonl"
+    sleep 15
+  done
+) &
+TELEMETRY_PID=$!
+"${VENV_ROOT}/bin/python" "${CONTROL_ROOT}/qwen30_exact_trace_client.py" \
+  --port "${PORT}" --requests-file "${REQUESTS_FILE}" --served-model "${SERVED_MODEL}" \
+  --output "${OUTPUT_ROOT}/results/result.json" --tpot-slo-ms 150 \
+  --timeout-seconds 5400
+kill "${TELEMETRY_PID}" 2>/dev/null || true
+TELEMETRY_PID=""
+curl -fsS "http://127.0.0.1:${PORT}/metrics" > "${OUTPUT_ROOT}/metrics/after.prom"
+INPUT_MANIFEST="${INPUT_MANIFEST}" OUTPUT_ROOT="${OUTPUT_ROOT}" \
+  "${VENV_ROOT}/bin/python" - <<'PY'
+import json
+import os
+from pathlib import Path
+
+manifest = json.loads(Path(os.environ["INPUT_MANIFEST"]).read_text())
+result = json.loads((Path(os.environ["OUTPUT_ROOT"]) / "results/result.json").read_text())
+if result["contract"]["row_vector_sha256"] != manifest["paired_row_vector_sha256"]:
+    raise SystemExit("real/sim paired row-vector digest mismatch")
+if result["contract"]["requests"] != manifest["requests"]:
+    raise SystemExit("real/sim request-count mismatch")
+if not all(request["success"] for request in result["requests"]):
+    raise SystemExit("one or more requests failed")
+PY
+
+cleanup
+nvidia-smi --query-gpu=index,name,uuid,driver_version,memory.total,memory.used,utilization.gpu \
+  --format=csv,noheader,nounits > "${OUTPUT_ROOT}/provenance/gpus.after.csv"
+find "${OUTPUT_ROOT}" -type f ! -path '*/provenance/artifacts.sha256' -print0 \
+  | sort -z | xargs -0 sha256sum > "${OUTPUT_ROOT}/provenance/artifacts.sha256"
+echo "FULL_REAL_COMPLETE rho=${RHO} config=${CONFIG} trial=${TRIAL}"