diff --git a/analysis/overnight_work_report.md b/analysis/overnight_work_report.md index 11e1c41..8b0f6d1 100644 --- a/analysis/overnight_work_report.md +++ b/analysis/overnight_work_report.md @@ -1,10 +1,13 @@ -# Overnight Work Report (2026-05-22) +# Overnight Work Report (2026-05-22, updated 2026-05-22 afternoon) ## Summary -Investigated routing optimization for agentic workloads on PD-combined serving. Found that **session-sticky routing with load-aware override** achieves the best balance of KV cache reuse (APC) and request latency. +Investigated routing optimization and selective PD offloading for agentic workloads. Found that **PD-combined with hybrid routing** (session-sticky + load-aware override) is strictly optimal for single-machine serving. All forms of KV transfer (full PD-sep, selective offload) add overhead that exceeds the isolation benefit. -**Key result**: +4.7pp APC improvement (44.7% → 49.4%) with zero latency regression. +**Key results**: +- Hybrid routing: +4.7pp APC (44.7% → 49.4%) with zero latency regression +- Adaptive v2 (selective Mooncake offload for HEAVY requests): +36% TTFT, +35% E2E — worse +- **Definitive conclusion: on single-machine 8 GPU, no KV transfer scheme helps agentic workloads** ---