gahow/agentic-kvc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
49 Commits 1 Branch 0 Tags
fc445df0ad0955377fd8cc63936662e9b7384faf
Commit Graph

2 Commits

Author SHA1 Message Date
Gahow Wang
a65ec42467 Update report: adaptive v2 confirms no KV transfer helps single-machine 
All PD/offload schemes tested are worse than PD-combined + hybrid routing:
  Combined hybrid:    TTFT=0.737  TPOT90=0.072  APC=49.4%  (BEST)
  PD-Sep 4P+4D:       TTFT=1.994  TPOT90=0.075  APC=40.2%
  Adaptive v2 offload: TTFT=1.462  TPOT90=0.077  APC=~45%

Definitive: single-machine agentic serving = PD-combined + smart routing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-22 10:15:08 +08:00
Gahow Wang
795edc6c66 Overnight work report: routing optimization achieves +4.7pp APC 
Summary of overnight autonomous session:
- Analyzed agentic workload patterns (91% KV reuse is intra-session)
- Simulated cache policies (LRU near-optimal, routing is the bottleneck)
- Implemented hybrid routing (session-sticky + load-aware override)
- Result: APC 44.7% -> 49.4% with zero latency regression

Key insight: routing quality > cache policy > PD separation for
single-machine agentic workloads.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-22 02:54:48 +08:00