aituner/runs/frontier-split-rootcause-v0/frozen-inputs/q235-ablation-a1/comparison.md

Qwen235 vLLM 0.20 Frontier vs real

case	metric	Frontier winner	real winner	match	regret	pair agreement
fixed-pd	ttft_mean_ms	tp8_ep8_mns128	tp8_ep8_mns128	yes	0.0%	100.0%
fixed-pd	ttft_p90_ms	tp8_ep8_mns128	tp8_ep8_mns64	no	0.2%	100.0%
fixed-pd	tpot_mean_ms	tp8_ep8_mns128	tp4_ep1_mns128	no	33.0%	0.0%
fixed-pd	tpot_p90_ms	tp8_ep8_mns128	tp4_ep1_mns128	no	37.2%	0.0%
fixed-pd	e2e_mean_ms	tp8_ep8_mns128	tp4_ep1_mns128	no	30.7%	0.0%
fixed-pd	e2e_p90_ms	tp8_ep8_mns128	tp4_ep1_mns128	no	34.6%	0.0%
fixed-po	ttft_mean_ms	tp8_ep8_mns128	tp8_ep8_mns128	yes	0.0%	100.0%
fixed-po	ttft_p90_ms	tp8_ep8_mns128	tp8_ep8_mns128	yes	0.0%	100.0%
fixed-po	e2e_mean_ms	tp8_ep8_mns128	tp8_ep8_mns128	yes	0.0%	100.0%
fixed-po	e2e_p90_ms	tp8_ep8_mns128	tp8_ep8_mns128	yes	0.0%	100.0%
trace-pd	ttft_mean_ms	tp8_ep8_mns128	tp8_ep8_mns128	yes	0.0%	66.7%
trace-pd	ttft_p90_ms	tp8_ep8_mns128	tp8_ep8_mns128	yes	0.0%	66.7%
trace-pd	tpot_mean_ms	tp8_ep8_mns64	tp8_ep8_mns64	yes	0.0%	66.7%
trace-pd	tpot_p90_ms	tp8_ep8_mns64	tp8_ep8_mns64	yes	0.0%	66.7%
trace-pd	e2e_mean_ms	tp8_ep8_mns128	tp8_ep8_mns64	no	0.6%	83.3%
trace-pd	e2e_p90_ms	tp8_ep8_mns128	tp8_ep8_mns64	no	6.2%	83.3%
trace-po	ttft_mean_ms	tp8_ep8_mns64	tp8_ep8_mns128	no	1.4%	66.7%
trace-po	ttft_p90_ms	tp8_ep8_mns128	tp8_ep8_mns64	no	0.3%	100.0%
trace-po	e2e_mean_ms	tp8_ep8_mns64	tp8_ep8_mns128	no	1.4%	66.7%
trace-po	e2e_p90_ms	tp8_ep8_mns128	tp8_ep8_mns64	no	0.3%	100.0%