aituner/docs/harness-ablation/qwen235b-prefill-2x2-progress-20260623.md

# Qwen235B prefill 2x2 progress - 2026-06-23

Snapshot: 2026-06-23 18:24 CST / 10:24 UTC.

本文整理当前 dash1/dash2/dash3 上的 Qwen235B prefill 2x2 实验进度。这个
case 仍在跑 strong-model arm，因此本文是 progress report，不是最终 aggregate
结论。

## 当前远端状态

| Host | 当前状态 | 说明 |
| --- | --- | --- |
| dash1 | running | `aituner-q235b-2x2-gpt55-20260623T010038Z` 仍在跑，当前是 `gpt-5.5 + naive` 的 trial-0004；8 张 H20 被 vLLM 占用。 |
| dash2 | idle | 没有 tmux/GPU 任务；最近完成的是 `qwen235b-prefill-jointprobe-harness-dash2-20260622T132010Z` harness-only 验证。 |
| dash3 | idle | 没有 tmux/GPU 任务；`gpt-5.4-mini` 2x2 arm 已完成并生成 report。 |

注意：三台机器共享 `/home/admin/cpfs/wjh/aituner/aituner`，所以 `.aituner` 和
`.aituner-reports` 在不同 dash 节点上看到的是同一批产物。

## 已完成：gpt-5.4-mini 2x2 arm

Report:

```text
.aituner-reports/qwen235b-prefill-2x2-gpt54mini-dash3-20260623T010038Z/report.md
```

Aggregate:

| Arm | Kind | Trials | Final req/s/GPU | Final/ref | TTT | AUC | Failed | No feasible |
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| `harness` | harness | 8 | 0.3217 | 1.0000 | 3 | 0.9483 | 0 | 1 |
| `naive` | naive | 8 | - | - | - | 0.0000 | 2 | 8 |

Interpretation:

- `gpt-5.4-mini + harness` 找到了 `0.3217 req/s/GPU`，达到该 report 的
  reference best。
- `gpt-5.4-mini + naive` 8 个 trials 都没有找到 feasible config，其中 2 个是
  engine launch failure。
- Report 中 `Harness-vs-naive pass/checks: 0/1` 是 aggregator 对
  `best_naive_final_per_gpu = null` 的保守处理：因为 naive 没有 feasible best，
  final ratio 无法计算，所以 pass 记为 false。就实际 tuning 结果而言，这个 arm
  是 harness dominates naive。

Harness trajectory:

| Trial | Patch | req/s/GPU | Pass rate | 说明 |
| ---: | --- | ---: | ---: | --- |
| 1 | `TP=8, DP=1` | 0.2879 | 0.9522 | 初始 topology 满足 SLO，但未达到最终 best。 |
| 2 | `TP=8, max-num-seqs=96` | 0.2879 | 0.9537 | 单独调 `max-num-seqs` 无明显提升。 |
| 3 | `TP=8, max-num-batched-tokens=16384, max-num-seqs=96` | 0.3085 | 0.9568 | joint runtime probe 提升。 |
| 4 | `TP=8, max-num-seqs=144, max-num-batched-tokens=32768` | 0.2879 | 0.9530 | 过大的 batching/seq 组合回退。 |
| 5 | `TP=4, DP=2` | - | - | 无 feasible best，说明 DP-heavy/mixed topology 不解决该 prefill path。 |
| 6 | `TP=8, max-num-seqs=96, max-num-batched-tokens=24576` | 0.2708 | 0.9523 | batching 进一步增大后回退。 |
| 7 | `TP=4, DP=1, max-num-seqs=96, max-num-batched-tokens=16384` | 0.2338 | 0.9590 | 少用 GPU 的 TP4/DP1 per-GPU 不占优。 |
| 8 | `TP=8, DP=1, max-num-seqs=128, max-num-batched-tokens=16384` | 0.3217 | 0.9508 | 当前 best。 |

这个结果说明：在 Qwen235B prefill case 上，harness 的价值不只是 topology
选择，还包括在 TTFT/prefill 方向下做受约束的 runtime joint probe。最终 best 是
`TP=8, DP=1, max-num-seqs=128, max-num-batched-tokens=16384`。

## 正在运行：gpt-5.5 2x2 arm

Session:

```text
tmux: aituner-q235b-2x2-gpt55-20260623T010038Z
driver log: .aituner/qwen235b-prefill-2x2-gpt55-dash1-20260623T010038Z.driver.log
```

Driver timeline:

```text
harness clean pair start 2026-06-23T01:00:40+00:00
harness clean pair done  2026-06-23T08:21:13+00:00
naive clean pair start   2026-06-23T08:21:13+00:00
```

Harness side has completed all 8 trials:

| Trial | Patch | req/s/GPU | Pass rate |
| ---: | --- | ---: | ---: |
| 1 | `TP=8, DP=1` | 0.2879 | 0.9522 |
| 2 | `TP=8, max-num-seqs=96` | 0.2879 | 0.9530 |
| 3 | `TP=8, max-num-batched-tokens=16384, max-num-seqs=96` | 0.3085 | 0.9561 |
| 4 | `TP=8, max-num-batched-tokens=32768, max-num-seqs=144` | 0.2783 | 0.9543 |
| 5 | `TP=8, DP=1, max-num-batched-tokens=24576, max-num-seqs=96` | 0.2654 | 0.9513 |
| 6 | `TP=4, DP=2, max-num-batched-tokens=16384, max-num-seqs=96` | - | - |
| 7 | `TP=8, DP=1, max-num-batched-tokens=16384, max-num-seqs=80` | 0.3156 | 0.9505 |
| 8 | `TP=8, max-num-batched-tokens=32768, max-num-seqs=120` | 0.2879 | 0.9508 |

Current harness best: `trial-0007`, `0.3156 req/s/GPU`.

Naive side is still running. Current state:

- Completed/recorded through trial-0003, with current best `0.2879 req/s/GPU`.
- trial-0004 is active with `TP=8, DP=1, max-num-batched-tokens=8192,
  max-num-seqs=128`.
- trial-0004 probe history so far:

| threshold | request rate | req/s/GPU | pass rate | feasible | main failures |
| ---: | ---: | ---: | ---: | --- | --- |
| 0.0625 | 1.5750 | 0.1969 | 0.9651 | true | TTFT misses and TTFT threshold violations |
| 0.09375 | 2.3650 | 0.2956 | 0.7308 | false | `slo_pass_rate_unrecoverable`, TTFT violations |
| 0.078125 | 1.9567 | 0.2446 | 0.9591 | true | TTFT misses and TTFT threshold violations |
| 0.0859375 | 2.1667 | 0.2708 | 0.9546 | true | TTFT misses and TTFT threshold violations |

As of the snapshot, vLLM is still processing requests for trial-0004, so the naive
side has not produced its final result or report yet.

## Prior Qwen235B context

These earlier runs explain why the current 2x2 matters:

| Run | Result | What it showed |
| --- | --- | --- |
| `qwen235b-prefill-clean-gpt55-dash1-20260621T160712Z` | harness 0.2879, naive 0.3217 | Earlier harness stopped/refined too weakly; naive found better final config. |
| `qwen235b-prefill-seqguard-gpt55-dash1-20260622T064445Z` | harness 0.2879, naive 0.2577 | Seq guard prevented the worst early-stop failure but still did not reach the old naive best. |
| `qwen235b-prefill-jointprobe-harness-dash2-20260622T132010Z` | harness-only 0.3085 | Joint `max-num-batched-tokens + max-num-seqs` probe improved over seqguard. |
| `qwen235b-prefill-2x2-gpt54mini-dash3-20260623T010038Z` | harness 0.3217, naive no feasible | Weak model plus harness now reaches the old best and dominates weak naive. |

The current evidence points to the harness needing both:

1. topology discipline: stay on `TP=8, DP=1` for this prefill-heavy 235B setup;
2. runtime joint probing: tune `max-num-batched-tokens` and `max-num-seqs` together
   instead of stopping after the first feasible TP8 result.

## Open item

The final Qwen235B 2x2 conclusion is blocked on the still-running
`gpt-5.5 + naive` arm on dash1. Once it completes, generate an aggregate report
combining:

- `qwen235b-prefill-2x2-gpt55-dash1-20260623T010038Z`
- `qwen235b-prefill-2x2-gpt54mini-dash3-20260623T010038Z`

and then update this progress report into a final ablation report.