48 lines
1.8 KiB
Markdown
48 lines
1.8 KiB
Markdown
# Qwen235 Fixed-PD state diagnosis
|
|
|
|
**Verdict:** Active-batch-count mismatch alone is insufficient: after exact reweighting to each config's real Running histogram, Frontier still predicts TP8 faster.
|
|
|
|
| Config | Real Running proxy mean | Sim decode batch mean | Proxy coverage |
|
|
|---|---:|---:|---:|
|
|
| tp4_ep1_mns64 | 4.667 | 13.537 | 100.0% |
|
|
| tp8_ep8_mns64 | 12.238 | 18.938 | 100.0% |
|
|
|
|
## Frontier internal component contrast over its own executed composition
|
|
|
|
Decode-token-weighted over both pure-decode and mixed prefill/decode steps.
|
|
|
|
| Component | TP8 - TP4 (ms/decoded token step) |
|
|
|---|---:|
|
|
| total | -73.6733 |
|
|
| moe_compute | -45.2710 |
|
|
| attention | -13.6950 |
|
|
| tp_dp_communication | -12.7244 |
|
|
| moe_routing | -1.7759 |
|
|
| dense_mlp_compute | -0.2890 |
|
|
| ep_communication | +0.0821 |
|
|
| pipeline_communication | +0.0000 |
|
|
| runtime_overhead | +0.0000 |
|
|
|
|
## Frontier internal component contrast at real Running proxy
|
|
|
|
Positive means TP8 slower; negative means Frontier gives TP8 an advantage.
|
|
|
|
| Component | TP8 - TP4 (ms/step) |
|
|
|---|---:|
|
|
| total | -46.0760 |
|
|
| moe_compute | -26.3387 |
|
|
| tp_dp_communication | -12.1532 |
|
|
| attention | -6.2393 |
|
|
| moe_routing | -1.2405 |
|
|
| dense_mlp_compute | -0.1769 |
|
|
| ep_communication | +0.0726 |
|
|
| pipeline_communication | +0.0000 |
|
|
| runtime_overhead | +0.0000 |
|
|
|
|
## Interpretation boundary
|
|
|
|
- Real observed TPOT contrast: +6.9500 ms/token.
|
|
- vLLM 10-second Running is active requests, not per-iteration decode batch; context lengths and graph buckets are unavailable in frozen logs.
|
|
- Frontier ledger exposes additive execution components but no direct graph bucket/padding/launch-overhead field; graph effects remain folded into predictors.
|
|
- Component deltas identify where Frontier creates its own TP8 advantage; without real per-stage measurements they are not yet root-cause proof.
|