agentic-kvc

Go to file

Gahow Wang c7afdc5074 Ablation 2: fire-and-forget vs await-prefill scheduling

Added --fire-and-forget flag to cache_aware_proxy.py for async prefill dispatch.

Results on 6P+2D config:
  Await:  TTFT=1.48s  TPOT=0.066s  E2E=5.95s  94% success
  FnF:    TTFT=5.32s  TPOT=0.037s  E2E=11.9s  85% success

Fire-and-forget improves TPOT by 44% (pipeline overlap) but degrades
TTFT by 260% (decode internally waits for KV, less efficiently than
proxy-level await) and increases errors from KV race conditions.

Full 4-way ablation summary in analyze_ablations.py.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-05-21 23:02:42 +08:00

analysis

Agentic workload PD separation analysis with trace-driven benchmarks

2026-05-21 21:21:57 +08:00

replayer

Agentic workload PD separation analysis with trace-driven benchmarks

2026-05-21 21:21:57 +08:00

scripts

Ablation 2: fire-and-forget vs await-prefill scheduling

2026-05-21 23:02:42 +08:00

.gitignore

Agentic workload PD separation analysis with trace-driven benchmarks