Agentic workload PD separation analysis with trace-driven benchmarks

Systematic study of prefill-decode disaggregation for agentic LLM workloads
using production GLM-5.1 coder trace (2.1M requests, 71B input tokens).

Key findings:
- Cache-aware routing improves TPOT p90 by 15% and APC from 20.8% to 44.7%
  without PD separation, matching PD-Sep's decode isolation benefit
- PD separation adds +72% TTFT overhead (KV transfer) with no TPOT gain
  when using the same cache-aware scheduler
- Prefill remains compute-bound even at 95% KV cache reuse (AI >1000x
  vs decode AI <2), but absolute FLOPs drop 71% from cache hits
- For agentic MoE workloads, cache-aware routing > PD separation

Infrastructure:
- Trace sampler preserving session structure + hash_ids for prefix sharing
- Async trace replayer with streaming TTFT/TPOT/E2E measurement
- Unified cache-aware + token-level load-balanced global scheduler proxy
  supporting both PD-colocated and PD-disaggregated (Mooncake/RDMA) modes
- vLLM 0.18.1 scheduler patch for KV transfer abort race condition
- Roofline analysis tool for prefill/decode compute characterization

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

replayer/__main__.py

@@ -0,0 +1,55 @@
+"""CLI entry point: python -m replayer replay ..."""
+
+from __future__ import annotations
+
+import argparse
+import asyncio
+import logging
+from pathlib import Path
+
+from .replay import ReplayConfig, replay_trace
+
+
+def main() -> None:
+    p = argparse.ArgumentParser(description="Trace replayer for vLLM benchmarking")
+    p.add_argument("--trace", type=Path, required=True, help="Sampled trace JSONL")
+    p.add_argument("--output", type=Path, required=True, help="Output metrics JSONL")
+    p.add_argument("--endpoint", type=str, required=True,
+                   help="vLLM server URL (e.g. http://localhost:8000)")
+    p.add_argument("--model", type=str, default="default", help="Model name for API")
+    p.add_argument("--time-scale", type=float, default=1.0,
+                   help="Time compression (>1 = faster)")
+    p.add_argument("--max-inflight-sessions", type=int, default=32)
+    p.add_argument("--concurrency-limit", type=int, default=256)
+    p.add_argument("--request-timeout", type=float, default=600.0)
+    p.add_argument("--request-limit", type=int, default=None,
+                   help="Limit number of requests to replay")
+    p.add_argument("-v", "--verbose", action="store_true")
+    args = p.parse_args()
+
+    logging.basicConfig(
+        level=logging.DEBUG if args.verbose else logging.INFO,
+        format="%(asctime)s %(levelname)s %(name)s: %(message)s",
+    )
+
+    config = ReplayConfig(
+        trace_path=args.trace,
+        output_path=args.output,
+        endpoint_url=args.endpoint.rstrip("/"),
+        model_name=args.model,
+        time_scale=args.time_scale,
+        max_inflight_sessions=args.max_inflight_sessions,
+        concurrency_limit=args.concurrency_limit,
+        request_timeout_s=args.request_timeout,
+        request_limit=args.request_limit,
+    )
+
+    results = asyncio.run(replay_trace(config))
+    succeeded = sum(1 for r in results if r.error is None)
+    print(f"\nDone: {succeeded}/{len(results)} requests succeeded")
+    print(f"Metrics: {args.output}")
+    print(f"Summary: {args.output.with_suffix('.summary.json')}")
+
+
+if __name__ == "__main__":
+    main()