Gahow Wang 4089ffd63f Fix replay methodology: trace-driven dispatch, no artificial limits ...

The replayer was artificially limiting concurrency with --max-inflight-sessions
(semaphore) and --time-scale (time compression), producing unrealistically low
1 req/GPU load that masked prefill-decode interference.

Replayer changes:
- Remove session_sem and time_scale entirely
- Each request dispatched at its trace timestamp exactly
- Sessions still sequential (turn N+1 waits for turn N completion)
- If turn completes late, next turn fires immediately

Sampler changes:
- Add --sample-ratio for GPU-proportional session sampling
- Keep --target-requests for backwards compat
- No time compression (preserve original arrival pattern)

bench.sh: remove --time-scale and --max-inflight-sessions args

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-05-23 12:43:41 +08:00

analysis

Add comprehensive research findings document

2026-05-23 07:16:31 +08:00

patches

Add vLLM patches directory for version-controlled patch management

2026-05-22 00:26:14 +08:00

replayer

Fix replay methodology: trace-driven dispatch, no artificial limits

2026-05-23 12:43:41 +08:00

scripts

Fix replay methodology: trace-driven dispatch, no artificial limits

2026-05-23 12:43:41 +08:00

third_party/vllm

PS experiments + H4 cache-gate + GPU profiling + Mooncake elif→if fix

2026-05-23 02:14:37 +08:00

.gitignore

Phase 1 milestone: system-level analysis + reproducible report

2026-05-22 16:17:41 +08:00

pyproject.toml

Agentic workload PD separation analysis with trace-driven benchmarks

2026-05-21 21:21:57 +08:00

REPORT.md

Benchmark concurrency gap: 1 req/GPU is 10-15x below production

2026-05-23 12:16:20 +08:00

TODO.md

LMetric routing policy (OSDI'26) + A/B results vs linear baseline

2026-05-22 16:57:32 +08:00

Description

No description provided

48 MiB

Languages

Python 82.9%

Shell 17.1%