gahow/aituner
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Harness: gate gpu-mem-util/seqs-raise on 'no untested TP increase' (frontier-closed)

Browse Source 
The first gpt-5.5 verification run exposed a bug in the prior gate: topology_settled =
cur_tp>base_tp let gpu-memory-utilization fire on a TP2 incumbent (TP2>baseline TP1)
and preempt the still-open TP4 frontier -- the harness proposed TP2+gpu-mem-util=0.92
at iter 2 instead of climbing to TP4. The candidate path runs before the topology-
frontier check, so a score>=0.35 runtime candidate wins.

Fix: gate runtime micro-tuning (gpu-mem-util, raising max-num-seqs) on the TP frontier
being closed -- topology_settled = no untested _next_allowed_tp remains (respects GPU
count, so TP4 is the real ceiling on 6 GPUs). New regression test: TP2 incumbent with
TP4 reachable must climb TP and must NOT propose gpu-mem-util. 116 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Gahow Wang
2026-06-19 13:33:29 +08:00
parent 76cca89a43
commit b3156a382a
2 changed files with 112 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
src/aituner/harness.py
View File

@@ -1194,14 +1194,21 @@ def _runtime_candidate_actions(
topology_patch = _preserve_topology_patch(study, anchor_flags)
actions: list[dict[str, Any]] = []
base_tp = _parse_int_like(study.engine.base_flags.get("tensor-parallel-size"), default=1)
base_dp = _parse_int_like(study.engine.base_flags.get("data-parallel-size"), default=1)
cur_tp = _parse_int_like(anchor_flags.get("tensor-parallel-size"), default=base_tp)
cur_dp = _parse_int_like(anchor_flags.get("data-parallel-size"), default=base_dp)
cur_tp = _parse_int_like(anchor_flags.get("tensor-parallel-size"), default=1)
cur_dp = _parse_int_like(anchor_flags.get("data-parallel-size"), default=1)
# Topology-before-runtime: gpu-mem-util / raising max-num-seqs are micro-tuning that is
# only justified once topology has moved off the baseline. At the baseline a latency
# bottleneck must still be answered with a topology change, not a runtime tweak.
topology_settled = cur_tp > base_tp or cur_dp > base_dp
# only justified once no untested TP increase remains. At an intermediate TP (e.g. TP2
# while TP4 is still reachable and untried) a latency bottleneck must still be answered
# by climbing TP, not a runtime tweak -- otherwise runtime tuning preempts the frontier.
_next_tp = _next_allowed_tp(study, current_tp=cur_tp, current_dp=cur_dp)
tp_frontier_open = (
_next_tp is not None
and _config_signature(
{"env_patch": {}, "flag_patch": {"tensor-parallel-size": _next_tp}}
)
not in tested_signatures
)
topology_settled = not tp_frontier_open
if "max-num-batched-tokens" in tunable:
current_mbt = _parse_int_like(anchor_flags.get("max-num-batched-tokens"), default=0)