gahow/aituner
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Decompose good/bad selection split across frozen surfaces

Browse Source 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Gahow Wang
2026-07-20 12:05:20 +08:00
parent 18c0b25ae7
commit 39766141fb
54 changed files with 24465 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
runs/frontier-split-rootcause-v0/frozen-inputs/q235-fourcase-a0/comparison.md Normal file
View File

@@ -0,0 +1,24 @@
# Qwen235 vLLM 0.20 Frontier vs real
| case | metric | Frontier winner | real winner | match | regret | pair agreement |
|---|---|---|---|---:|---:|---:|
| fixed-pd | ttft_mean_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | no | 4.2% | 0.0% |
| fixed-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 0.2% | 100.0% |
| fixed-pd | tpot_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 33.0% | 0.0% |
| fixed-pd | tpot_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 37.2% | 0.0% |
| fixed-pd | e2e_mean_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 30.7% | 0.0% |
| fixed-pd | e2e_p90_ms | tp8_ep8_mns128 | tp4_ep1_mns128 | no | 34.6% | 0.0% |
| fixed-po | ttft_mean_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | no | 5.9% | 0.0% |
| fixed-po | ttft_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | no | 1.7% | 0.0% |
| fixed-po | e2e_mean_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | no | 5.9% | 0.0% |
| fixed-po | e2e_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns128 | no | 1.7% | 0.0% |
| trace-pd | ttft_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 66.7% |
| trace-pd | ttft_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns128 | yes | 0.0% | 66.7% |
| trace-pd | tpot_mean_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | yes | 0.0% | 66.7% |
| trace-pd | tpot_p90_ms | tp8_ep8_mns64 | tp8_ep8_mns64 | yes | 0.0% | 66.7% |
| trace-pd | e2e_mean_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 0.6% | 83.3% |
| trace-pd | e2e_p90_ms | tp8_ep8_mns128 | tp8_ep8_mns64 | no | 6.2% | 66.7% |
| trace-po | ttft_mean_ms | tp4_ep1_mns64 | tp8_ep8_mns128 | no | 7.0% | 0.0% |
| trace-po | ttft_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns64 | no | 21.2% | 0.0% |
| trace-po | e2e_mean_ms | tp4_ep1_mns64 | tp8_ep8_mns128 | no | 7.0% | 0.0% |
| trace-po | e2e_p90_ms | tp4_ep1_mns128 | tp8_ep8_mns64 | no | 21.2% | 0.0% |