gahow/agentic-kvc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Commit Graph

  • 8829928fc5 Cache-size sweep: build_meta is O(|cache|), +85.6 μs / 1k blocks Gahow Wang 2026-05-26 23:34:21 +08:00
  • 54de78eb11 Connector tax RESULTS.md: errata + run-to-run variance disclosure Gahow Wang 2026-05-26 23:33:01 +08:00
  • e3480f7d28 8-instance connector tax: +2% at non-saturated, +17% only at saturation Gahow Wang 2026-05-26 21:32:46 +08:00
  • c8ec73c548 Connector tax: high-concurrency confirms +7-9% tax, resolves trace-replay gap Gahow Wang 2026-05-26 21:00:25 +08:00
  • a473c71cac Connector tax Phase A: build_connector_meta is 1.4ms/step (the tax source) Gahow Wang 2026-05-26 19:33:15 +08:00
  • 297fed6e73 Microbench 3 (connector_tax): infrastructure for KV connector substrate tax Gahow Wang 2026-05-26 17:27:41 +08:00
  • 3fdcec9c0f Fix review P2s: lockfile, model path convention, trap robustness Gahow Wang 2026-05-26 16:05:43 +08:00
  • dc6d24d1ca Add NIXL substrate isolation control + attribution decomposition Gahow Wang 2026-05-26 16:02:12 +08:00
  • 645b067dd4 Fix review bugs: PD-sep counter leaks, hardcoded paths, missing deps Gahow Wang 2026-05-26 15:54:55 +08:00
  • 0eb49dcc34 Fix NIXL multi-instance port conflict: per-instance SIDE_CHANNEL_PORT Gahow Wang 2026-05-26 15:09:16 +08:00
  • 151bf33541 Add unified_nixl_both policy: NIXL connector isolation control Gahow Wang 2026-05-26 14:57:54 +08:00
  • 06dd175441 Microbench 1 plots: prefill-decode interference heatmap + lines Gahow Wang 2026-05-26 14:21:30 +08:00
  • 72790ae6c1 PD-sep server-side profiling: vLLM patches + per-request breakdown Gahow Wang 2026-05-26 13:59:09 +08:00
  • d76eb02637 Elastic migration v2 section: PD-sep on agentic workload is net negative Gahow Wang 2026-05-26 13:28:37 +08:00
  • 95c8ef853c Fix proxy shadow drift: actively reconcile against vLLM /metrics Gahow Wang 2026-05-26 11:29:02 +08:00
  • 4b833d33b7 unified_v2.1: relax gates + add unified_kv_both isolation control Gahow Wang 2026-05-26 10:40:57 +08:00
  • 19f69a9d2e unified_v2: selective per-request PD-sep via Mooncake (E3+E4) Gahow Wang 2026-05-26 09:25:45 +08:00
  • c63dc151a0 Agentic PD / Unified routing story plan draft Gahow Wang 2026-05-26 01:12:42 +08:00
  • 0881942cf3 Window 1 results: recompute with fixed metrics + reframe limitations Gahow Wang 2026-05-26 01:08:55 +08:00
  • 0e82612100 Fix B3 analysis bugs from subagent audit (median + percentile + sweep) Gahow Wang 2026-05-26 01:08:37 +08:00
  • 8ac41a8684 Agentic dispatch coupling: trace-replay session-sequentiality is realistic Gahow Wang 2026-05-26 01:00:25 +08:00
  • f784e49c07 Microbench: prefill-decode interference + PD transfer lifecycle Gahow Wang 2026-05-26 00:57:06 +08:00
  • 559faa1e26 B2 finding: TPOT idx peaks at 32k, not 65k — cost migrates to TTFT Gahow Wang 2026-05-26 00:35:45 +08:00
  • 4722883903 Audit package refresh: Window 1 supported claims + risk register Gahow Wang 2026-05-25 23:25:27 +08:00
  • 0c3220cbb8 Window 1 results: combined B1' + B2 + B3 report and artifacts Gahow Wang 2026-05-25 23:25:09 +08:00
  • b7902061d1 Window 1 analysis: APC upper bound, B2 window-overlap, figure renderer Gahow Wang 2026-05-25 23:24:54 +08:00
  • b9f324f2e6 B2 interference driver: request return_token_ids + text fallback Gahow Wang 2026-05-25 22:39:54 +08:00
  • df3249925b B3 analyze: prefer per-policy engine_state over slicing shared dir Gahow Wang 2026-05-25 22:19:43 +08:00
  • 1d87082ca1 B3: cold-start isolated policy runner (clean APC per cell) Gahow Wang 2026-05-25 20:33:44 +08:00
  • 08530b3915 B3 policies: pseudocode reference for the five-policy sweep Gahow Wang 2026-05-25 19:57:02 +08:00
  • 123a74a4b9 B3 report renderer: incremental markdown table from comparison JSON Gahow Wang 2026-05-25 18:58:21 +08:00
  • 92db1c4370 B3 post-run helpers: engine_state slicer + per-policy aggregator Gahow Wang 2026-05-25 18:51:33 +08:00
  • e23128ad65 B2: PD-colo interference microbench harness + sweep aggregator Gahow Wang 2026-05-25 17:54:51 +08:00
  • c6b7c3471b B3: load_only + sticky policies, capped-trace builder, sweep driver Gahow Wang 2026-05-25 17:54:24 +08:00
  • 763355b825 A5 fix: worker-id resolution and vLLM cmpl- rid stripping Gahow Wang 2026-05-25 16:47:23 +08:00
  • cd82b8c2a2 PD-sep matrix results: C2/C3/C4 figures + empirical mechanism refined Gahow Wang 2026-05-25 16:23:52 +08:00
  • 25445e3d18 A5: joined analysis with reuse decomp, interference, hot-spot, labels Gahow Wang 2026-05-25 16:19:33 +08:00
  • f42c715ec1 A4: open-loop session-causal SRR loadgen Gahow Wang 2026-05-25 16:19:20 +08:00
  • 5816aad731 A3: vLLM scheduler patch for step-level JSONL log Gahow Wang 2026-05-25 16:19:11 +08:00
  • fe556b5d98 A2: proxy worker-state snapshot and request-id passthrough Gahow Wang 2026-05-25 16:19:01 +08:00
  • d57e338366 A1: replayer instrumentation for cross-process join Gahow Wang 2026-05-25 16:18:52 +08:00
  • e5761fa6f3 Characterization plan: progress snapshot + Claude work plan Gahow Wang 2026-05-25 16:18:41 +08:00
  • 5ed6f6fe5b Add characterization result figures Gahow Wang 2026-05-25 15:15:10 +08:00
  • 0f64fb3261 Add agentic workload characterization audit scaffold Gahow Wang 2026-05-25 15:01:18 +08:00
  • 21ffb3d4f7 PD-sep matrix infrastructure: bench.sh pdsep mode + matrix driver Gahow Wang 2026-05-25 11:47:33 +08:00
  • 4028c587b1 Paper section: system analysis + workload figures + KV-wall model Gahow Wang 2026-05-25 11:41:31 +08:00
  • d71a111099 Paper section: PD-sep scaffold + drop --enforce-eager from launch scripts Gahow Wang 2026-05-25 11:24:16 +08:00
  • 6a27f75337 Docs: reconcile routing docs with current hybrid direction Gahow Wang 2026-05-25 10:47:14 +08:00
  • ac6534c3ff Cleanup: retire dead PUSH path + extract hybrid picker Gahow Wang 2026-05-25 10:46:57 +08:00
  • 255c8e6884 Hybrid routing: LMetric for LB + explicit affinity for high-cache sessions Gahow Wang 2026-05-25 09:05:08 +08:00
  • 448361cf83 Update design doc: final results + review findings Gahow Wang 2026-05-25 03:48:18 +08:00
  • 4c583f2f1c Revert relaxed gate + push_cost fix: 134 offloads destroyed performance Gahow Wang 2026-05-25 03:38:59 +08:00
  • bf4469a150 Fix cost model: accurate push_cost + aligned hard gate Gahow Wang 2026-05-25 01:01:03 +08:00
  • 1d2148cf65 Remove second push_new gate that caused downgrade-to-cold-LOCAL Gahow Wang 2026-05-25 00:42:31 +08:00
  • 3ae99293fd Relax _push_allowed: gate on request size, not cache savings Gahow Wang 2026-05-25 00:03:28 +08:00
  • cc6e5625bb Revert Approach B (session migration): overhead exceeds LB benefit Gahow Wang 2026-05-24 23:43:47 +08:00
  • 5b1d36080a Fix B2 migration: correct offload call signature (c_inst/d_inst order + cache_hit arg) Gahow Wang 2026-05-24 22:46:46 +08:00
  • 5772149d36 Approach B v2: TTFT-based migration trigger Gahow Wang 2026-05-24 21:54:06 +08:00
  • 45b82272c3 Add migration policy design doc with A/B experiment results Gahow Wang 2026-05-24 18:24:49 +08:00
  • e9919605af Approach B: session-level lazy migration trigger Gahow Wang 2026-05-24 17:34:06 +08:00
  • e06de5144b Approach A: contention-aware cost model with migration discount Gahow Wang 2026-05-24 17:24:27 +08:00
  • e13391eeab Evict migrated blocks from prefix cache after KV send completes Gahow Wang 2026-05-24 16:56:34 +08:00
  • 4b50c5a08d Fix unified cost model: include decode load in queue + hard overload gate Gahow Wang 2026-05-24 16:25:02 +08:00
  • 9cebdb6b9b Fix multi-turn replay fidelity: track realized output tokens across all components Gahow Wang 2026-05-24 14:47:51 +08:00
  • cc4a9c91e7 Fix estimate_hit: reuse _lookup_by_tokens instead of reimplementing hash Gahow Wang 2026-05-24 12:41:53 +08:00
  • 657812f8c4 Add deploy_vllm_patches.sh: sync third_party/vllm patches to site-packages Gahow Wang 2026-05-24 11:59:52 +08:00
  • bf76273778 Add --offload-mode switch for ablation (direct_read vs cached_prefill) Gahow Wang 2026-05-24 11:24:15 +08:00
  • cdf83493ab Fix A+C: real cache sync + cached-prefill-on-C architecture Gahow Wang 2026-05-24 11:22:38 +08:00
  • 2b9eae0d54 Report §3.9: Unified routing final results — TTFT -25%, E2E -7% Gahow Wang 2026-05-24 03:15:32 +08:00
  • 97f4fe5164 Fix: rename inst->chosen in generate function (NameError crash) Gahow Wang 2026-05-24 02:55:01 +08:00
  • 5892739159 Add session affinity as soft preference in unified routing Gahow Wang 2026-05-24 02:37:58 +08:00
  • 6b255fad91 Unified routing: single argmin(expected_latency) over all instances Gahow Wang 2026-05-24 02:21:34 +08:00
  • 1cd0a18e2c Report §3.8: Document direct KV cache migration architecture + bugs fixed Gahow Wang 2026-05-24 01:52:38 +08:00
  • 8c267ec54e VERIFIED: Bootstrap-triggered PUSH works end-to-end! Gahow Wang 2026-05-24 01:50:37 +08:00
  • e3a1d70cf2 Switch from RDMA READ to bootstrap-triggered PUSH Gahow Wang 2026-05-24 01:47:49 +08:00
  • 6716a3401a Progress: hash matching FIXED (640/640), RDMA read returns -1 Gahow Wang 2026-05-24 01:40:52 +08:00
  • 0bb6a67ed3 Fix: use sha256 (default) not sha256_cbor for block hash computation Gahow Wang 2026-05-24 01:36:05 +08:00
  • 08d5e12838 Fix NONE_HASH import: use module ref instead of from-import (value binding bug) Gahow Wang 2026-05-24 01:32:19 +08:00
  • 7e91b83d88 Set PYTHONHASHSEED=42 for elastic mode to ensure consistent block hashes Gahow Wang 2026-05-24 01:27:52 +08:00
  • ee2301ae17 Fix: token lookup condition should check hash_table not block_pool Gahow Wang 2026-05-24 01:21:49 +08:00
  • 0c88609caa Fix: use synced hash table + sha256_cbor for token-based lookup (same process NONE_HASH) Gahow Wang 2026-05-24 01:18:47 +08:00
  • 0500350849 Fix hash mismatch: token-based lookup instead of cross-instance hash matching Gahow Wang 2026-05-24 01:14:33 +08:00
  • a1f30e5fce Add hash_table_sync logging + gap analysis Gahow Wang 2026-05-24 00:38:14 +08:00
  • 1cf03c6e79 Cost model: add interference penalty for co-located heavy prefill Gahow Wang 2026-05-23 23:59:06 +08:00
  • 29b901b145 Fix scheduler assertion crash on partial remote prefill finished_recving Gahow Wang 2026-05-23 23:33:26 +08:00
  • 4f93bb5b8a Report §3.8: Direct RDMA read results — HEAVY TTFT -70%, TPOT p90 -38% Gahow Wang 2026-05-23 22:56:16 +08:00
  • a7514fc3d5 Fix retry syntax: async generator can't use return, use break+try/finally Gahow Wang 2026-05-23 22:37:32 +08:00
  • daeb95eca0 RDMA overhead 2.0→0.1s (direct read is raw memory, not scheduler flow) + retry on ConnectError to handle kv_both connection instability Gahow Wang 2026-05-23 22:33:10 +08:00
  • 5c66f500fc Fix offload gate: remove cache_gate for direct RDMA read, fix cost model Gahow Wang 2026-05-23 22:01:43 +08:00
  • 23788f7cd5 Fix: import field from dataclasses for PullReqMeta Gahow Wang 2026-05-23 21:29:24 +08:00
  • 1dea82f2ff launch_phase1_ps: parameterise project + model paths (B6 followup) Gahow Wang 2026-05-23 21:14:15 +08:00
  • 52a54e44af proxy: split session_affinity per mode + vLLM patch self-check (M4, S2) Gahow Wang 2026-05-23 21:12:56 +08:00
  • c843f2e3db proxy: Settings dataclass + cache-ratio gate + P-pick offload penalty (B4, M2, M3, D5) Gahow Wang 2026-05-23 21:11:17 +08:00
  • 0701f84c00 tests: add minimal coverage for percentile + proxy routing (S1) Gahow Wang 2026-05-23 21:07:14 +08:00
  • 7c7f8b951a replayer: wire --max-inflight-sessions cap into replay loop (B2) Gahow Wang 2026-05-23 21:04:09 +08:00
  • 2c7f7fdaae replayer: restore optional max_inflight_sessions for backwards compat Gahow Wang 2026-05-23 21:02:26 +08:00
  • a7df84bd3b Direct RDMA read: D reads cached KV from C's GPU without C's scheduler Gahow Wang 2026-05-23 21:02:13 +08:00
  • 020be9f444 proxy: real LRU for cached_blocks + shadow-state reconcile loop (M1, M5) Gahow Wang 2026-05-23 21:00:35 +08:00
  • 0ed1ce200e metrics: replace round-based percentile with linear interpolation (B5) Gahow Wang 2026-05-23 21:00:24 +08:00
  • 0958823cdb REPORT: add §1.1 errata flagging superseded sections (S3) Gahow Wang 2026-05-23 20:58:38 +08:00