Document full config signature validation
This commit is contained in:
@@ -281,3 +281,92 @@ TP4 将 best req/s/GPU 从 1.0000 提高到 1.7375。
|
||||
P0 measurement/stop-order slice: passed.
|
||||
P0 full coverage-relative harness: not yet passed.
|
||||
```
|
||||
|
||||
## 2026-06-26 normalized full-config validation
|
||||
|
||||
Commit `48911b6` 修复了上一节暴露的新 blocker:no-repeat 不再只比较 patch
|
||||
signature,而是比较 normalized effective full config。
|
||||
|
||||
实现语义:
|
||||
|
||||
```text
|
||||
effective_config =
|
||||
normalize(base_envs + env_patch,
|
||||
base_flags + flag_patch)
|
||||
|
||||
no_repeat_signature = stable_json(effective_config)
|
||||
```
|
||||
|
||||
因此下面两个 proposal 在 validator 看来是同一个 full config:
|
||||
|
||||
```text
|
||||
baseline patch: {}
|
||||
noop patch: {"tensor-parallel-size": 8}
|
||||
```
|
||||
|
||||
本地验证:
|
||||
|
||||
```text
|
||||
PYTHONPATH=src python3 -m unittest discover -s tests
|
||||
Ran 145 tests OK
|
||||
```
|
||||
|
||||
dash1 validation:
|
||||
|
||||
```text
|
||||
run label = adversarial-badstart-fullsig-48911b6-20260626T133112Z
|
||||
git sha = 48911b658bbf052d70d952d1cdf55ad6b50ba7a5
|
||||
machine = dash1, 8x H20
|
||||
```
|
||||
|
||||
Spec 仍使用同一个 adversarial bad-start:
|
||||
|
||||
```text
|
||||
tensor-parallel-size = 8
|
||||
data-parallel-size = 1
|
||||
gpu-memory-utilization = 0.5
|
||||
max-num-seqs = 8
|
||||
search.auto_high.enabled = true
|
||||
LLM endpoint disabled
|
||||
```
|
||||
|
||||
结果:
|
||||
|
||||
| trial | proposal | best sampling_u | request_rate | req/s/GPU | pass |
|
||||
| --- | --- | ---: | ---: | ---: | ---: |
|
||||
| trial-0001 | baseline TP8, DP1, gmu0.5, mns8 | 0.935616858887 | 8.00 | 1.0000 | 1.0000 |
|
||||
| trial-0002 | `tensor-parallel-size=4` | 0.810867944369 | 6.95 | 1.7375 | 0.9832 |
|
||||
| trial-0003 | `tensor-parallel-size=4`, `gpu-memory-utilization=0.9` | 0.935616858887 | 8.00 | 2.0000 | 1.0000 |
|
||||
|
||||
关键 observation:
|
||||
|
||||
```text
|
||||
旧 trial-0003:
|
||||
{"tensor-parallel-size": 8}
|
||||
-> 等价于 baseline,但仍被执行
|
||||
|
||||
新 trial-0003:
|
||||
{"tensor-parallel-size": 4, "gpu-memory-utilization": 0.9}
|
||||
-> 在已验证 TP4 topology 上继续测试 KV/cache headroom
|
||||
```
|
||||
|
||||
这证明 normalized full-config signature 已经阻止了 patch-level no-op 重测。
|
||||
|
||||
机制解释:
|
||||
|
||||
1. baseline TP8 saturate search ceiling 只被记录为 measurement evidence;
|
||||
2. 因为 objective 是 `req/s/GPU`,topology/resource-efficiency contrast 仍未覆盖,所以
|
||||
validator 不允许 stop;
|
||||
3. harness 先测试相邻低 TP topology,TP4 把 `req/s/GPU` 从 `1.0` 提高到 `1.7375`;
|
||||
4. no-repeat 用 full config signature block 掉等价 TP8 patch;
|
||||
5. harness 在 settled TP4 topology 上继续测试 runtime headroom,`gmu=0.9` 把
|
||||
`req/s/GPU` 提高到 `2.0`。
|
||||
|
||||
当前 verdict 更新为:
|
||||
|
||||
```text
|
||||
P0 measurement/stop-order slice: passed.
|
||||
P0 normalized full-config no-repeat slice: passed.
|
||||
P0 single adversarial bad-start recovery: passed for this case.
|
||||
P0 distribution-level bad-start robustness: not yet proven.
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user