Gahow Wang 080a8fa138 Chunk-size ablation + comprehensive synthesis ...

max_num_batched_tokens sweep at 16 sessions (2048/4096/8192/16384):
- Default 8192 has best overall TPOT p90 (0.106) and E2E p50 (5.83)
- 16384: HEAVY TTFT -16%, HEAVY TPOT -17%, but overall worse (+18%)
- Smaller chunks (2048/4096) always worse (scheduler overhead)

bench.sh now supports --max-batched-tokens flag.

Updated elastic_hypotheses.md with H8 (high concurrency validated),
H9 (elastic RDMA at 16s rejected), and final synthesis.

Key conclusion: for agentic workloads, the dominant optimization is
cache-aware session-sticky routing (-60% TTFT, +24pp APC vs RR).
Neither PD-Sep, LMetric, elastic RDMA, nor chunk-size tuning provides
additional benefit beyond well-tuned routing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-05-23 07:15:02 +08:00

analysis

16-session contention: TPOT +45% from prefill-decode interference

2026-05-23 05:51:47 +08:00

patches

Add vLLM patches directory for version-controlled patch management

2026-05-22 00:26:14 +08:00

replayer

Elastic P2P offload: TTFT p50 -49% vs baseline (0.551 vs 1.080)

2026-05-22 13:50:25 +08:00

scripts

Chunk-size ablation + comprehensive synthesis

2026-05-23 07:15:02 +08:00

third_party/vllm

PS experiments + H4 cache-gate + GPU profiling + Mooncake elif→if fix

2026-05-23 02:14:37 +08:00

.gitignore

Phase 1 milestone: system-level analysis + reproducible report

2026-05-22 16:17:41 +08:00

pyproject.toml

Agentic workload PD separation analysis with trace-driven benchmarks

2026-05-21 21:21:57 +08:00

REPORT.md

Invalidate prior A/B results + add proper experiment harness

2026-05-22 17:54:21 +08:00

TODO.md

LMetric routing policy (OSDI'26) + A/B results vs linear baseline

2026-05-22 16:57:32 +08:00

Description

No description provided

48 MiB

Languages

Python 82.9%

Shell 17.1%