393 B
393 B
Qwen30 real latency case audit
Prefill-only: False.
| Objective | Real winner | Value (ms) |
|---|---|---|
| ttft_ms:pooled_mean_ms | tp4_mns32 | 123.16 |
| ttft_ms:pooled_p90_ms | tp4_mns64 | 126.84 |
| tpot_ms:pooled_mean_ms | tp4_mns8 | 8.42 |
| tpot_ms:pooled_p90_ms | tp4_mns8 | 8.77 |
| e2e_ms:pooled_mean_ms | tp1_mns32 | 3203.72 |
| e2e_ms:pooled_p90_ms | tp1_mns32 | 3389.19 |