30 lines
1.6 KiB
Markdown
30 lines
1.6 KiB
Markdown
# Frontier piecewise graph-profile vs. real vLLM
|
|
|
|
Each cell uses its TP-normalized, 129-request trace. Frontier values are one deterministic simulation; real values pool three fresh-server trials (387 requests/cell).
|
|
|
|
| Config | TTFT sim / real mean (ms) | TPOT sim / real mean (ms) | E2E sim / real mean (ms) |
|
|
|---|---:|---:|---:|
|
|
| tp1_mns8 | 1669003.8 / 88479.9 | 76.0 / 14.3 | 1930772.9 / 137996.4 |
|
|
| tp1_mns16 | 870562.2 / 17378.2 | 99.1 / 20.6 | 1209178.7 / 87355.2 |
|
|
| tp1_mns32 | 316980.6 / 620.5 | 119.6 / 23.8 | 722298.9 / 81051.2 |
|
|
| tp1_mns64 | 105003.3 / 619.8 | 137.8 / 24.1 | 549486.0 / 82253.7 |
|
|
| tp2_mns8 | 1171654.6 / 95920.1 | 51.1 / 9.2 | 1348614.2 / 128296.9 |
|
|
| tp2_mns16 | 606885.2 / 34443.0 | 64.4 / 14.1 | 828082.5 / 82936.7 |
|
|
| tp2_mns32 | 224495.4 / 1050.6 | 74.8 / 18.0 | 479340.1 / 62267.5 |
|
|
| tp2_mns64 | 30524.5 / 375.8 | 82.6 / 18.2 | 311160.1 / 62353.6 |
|
|
| tp4_mns8 | 834500.2 / 102605.9 | 35.4 / 6.8 | 957838.5 / 126405.4 |
|
|
| tp4_mns16 | 429098.7 / 35042.6 | 43.2 / 8.8 | 578209.7 / 65779.1 |
|
|
| tp4_mns32 | 166734.5 / 6296.2 | 49.4 / 12.2 | 336309.9 / 47944.5 |
|
|
| tp4_mns64 | 26688.5 / 246.0 | 53.7 / 13.2 | 209788.3 / 44985.1 |
|
|
|
|
## Selection agreement
|
|
|
|
| Target | Frontier winner | Real winner | Match | Pairwise agreement |
|
|
|---|---|---|---:|---:|
|
|
| ttft_ms:mean | tp4_mns64 | tp4_mns64 | yes | 84.8% (56/66) |
|
|
| ttft_ms:p90 | tp4_mns64 | tp4_mns64 | yes | 84.8% (56/66) |
|
|
| tpot_ms:mean | tp4_mns8 | tp4_mns8 | yes | 97.0% (64/66) |
|
|
| tpot_ms:p90 | tp4_mns8 | tp4_mns8 | yes | 97.0% (64/66) |
|
|
| e2e_ms:mean | tp4_mns64 | tp4_mns64 | yes | 92.4% (61/66) |
|
|
| e2e_ms:p90 | tp4_mns64 | tp4_mns64 | yes | 83.3% (55/66) |
|