Approach B v2: TTFT-based migration trigger

Replace num_requests threshold with recent TTFT median as migration
trigger. Track per-instance rolling TTFT (last 8 requests) and trigger
migration when median > 5s (configurable). Target is the instance with
lowest recent TTFT, requiring > 2x improvement to justify migration.

This is more responsive than the instantaneous num_requests signal
because TTFT directly measures the user-facing impact of contention.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

FIXES.md

@@ -0,0 +1,768 @@
+# Repo 修复指南 (FIXES.md)
+
+> 本文档对应 2026-05-23 的 repo review。每条 issue 自包含：定位、动机、复现/验证、改法。按严重度从高到低排列，建议**自上而下**逐项修复，每条修完独立提交一个 commit。
+
+---
+
+## 目录
+
+- [B1. 删除死状态 `_inst_cumulative_tokens`](#b1)
+- [B2. 修复 replayer CLI 与 shell 脚本不一致（阻断实验）](#b2)
+- [B3. 处理 PD-sep `--fire-and-forget` 损坏路径](#b3)
+- [B4. 实现或移除 H4 cache-ratio gate](#b4)
+- [B5. 修复 `_percentile` off-by-one](#b5)
+- [B6. 统一 `bench.sh` 的模型路径](#b6)
+- [M1. `cached_blocks` 替换策略改为真正的 LRU](#m1)
+- [M2. P 候选选择避开 `active_p_offloads`](#m2)
+- [M3. 把 `MAX_OFFLOAD_INFLIGHT` 暴露为 CLI 参数](#m3)
+- [M4. `session_affinity` 在 combined / pd-sep 之间命名空间隔离](#m4)
+- [M5. fallback 路径 client 断流时的资源泄漏](#m5)
+- [M6. `_send_prefill_async` 与同步路径的核算不一致](#m6)
+- [D1. 移除 `_send_prefill_async` 与 `--fire-and-forget`](#d1)
+- [D2. 删除/归档 `run_benchmark.sh` 与 `run_experiments.sh`](#d2)
+- [D3. 归档历史一次性 `analyze_*.py` / `compare_*.py`](#d3)
+- [D4. 修正 `compute_roofline.py` 的硬编码 trace 路径](#d4)
+- [D5. `HEAVY_THRESHOLD` / `OVERLOAD_FACTOR` 改读 args](#d5)
+- [S1. 给 `replayer/metrics.py` 与 cost-model 加单元测试](#s1)
+- [S2. 给 vLLM patch 加 import-time 校验](#s2)
+- [S3. REPORT.md 加 errata block](#s3)
+- [验收清单](#验收清单)
+
+---
+
+<a id="b1"></a>
+## B1. 删除死状态 `_inst_cumulative_tokens`
+
+**严重度**: High（误导性死代码）。
+
+**定位**: `scripts/cache_aware_proxy.py:76, 102–104, 125`。
+
+**问题**:
+- `_inst_cumulative_tokens` 是 module-level list，每次 turn 1 路由后 `+= input_length`。
+- 全 repo grep 这个名字只有写入点，没有任何读取。
+
+**验证**:
+```bash
+grep -rn "_inst_cumulative_tokens" /home/gahow/phd/agentic-kv
+# 只应该看到 cache_aware_proxy.py 自己的 5 行；如有其它读取者，先确认意图再删
+```
+
+**改法**:
+1. 删除 `cache_aware_proxy.py:76` 行 `_inst_cumulative_tokens: list[int] = []`。
+2. 删除 `pick_instance` 内的 `global _inst_cumulative_tokens` 与 `:103-104` 的初始化。
+3. 删除 `:125` 的累加。
+4. 不需要替代实现——load 计算用 `inst.ongoing_tokens`，session 粘性用 `affinity` dict。
+
+---
+
+<a id="b2"></a>
+## B2. 修复 replayer CLI 与 shell 脚本不一致（最高优先级）
+
+**严重度**: Critical（阻断 REPORT 自己规定的 next-step 实验）。
+
+**定位**:
+- `replayer/__main__.py:14-26`: argparse 当前**只**接受 `--trace --output --endpoint --model --concurrency-limit --request-timeout --request-limit -v`。
+- `scripts/run_benchmark.sh:32, 70-71`: 仍传 `--time-scale` 和 `--max-inflight-sessions`。
+- `scripts/run_experiments.sh:58-59`: 同样问题。
+- `REPORT.md:521, 541`: 把 `--max-inflight-sessions 64+` 列为 next step。
+
+**问题**:
+- 跑这两个 shell 脚本会立刻 `SystemExit(2)`：unrecognized arguments。
+- 报告里的"下一步实验"无法执行。
+
+**决策**: 两条路线，**二选一**，本 repo 推荐路线 A。
+
+### 路线 A（推荐）：恢复 `--max-inflight-sessions`，保持 `--time-scale` 移除
+
+理由：REPORT §3.6 已经论证 trace-driven replay（无时间压缩）是正确的；但高并发实验需要一个并发上限旋钮。把 `--max-inflight-sessions` 重新加回来，语义为"全局活跃 session 数上限的 semaphore"。
+
+**改法**:
+
+1. 修改 `replayer/__main__.py`：
+   ```python
+   p.add_argument("--max-inflight-sessions", type=int, default=None,
+                  help="Cap concurrent active sessions (None = unlimited; "
+                       "use to simulate higher-than-trace concurrency)")
+   ```
+   并把它塞进 `ReplayConfig`：
+   ```python
+   config = ReplayConfig(
+       ...
+       max_inflight_sessions=args.max_inflight_sessions,
+   )
+   ```
+
+2. 修改 `replayer/replay.py` 的 `ReplayConfig` 与 dispatch 逻辑：
+   - 在 `ReplayConfig` 里加 `max_inflight_sessions: int | None = None`。
+   - 在 `replay_trace` 里：若 `max_inflight_sessions` 不为 None，创建 `asyncio.Semaphore(max_inflight_sessions)`，每个 session 任务 `async with sem:` 包住整段 session 重放（不是单个 request）。
+   - 若为 None，保持现有行为（无上限，仅 `concurrency_limit` 是 HTTP 层 safety semaphore）。
+
+3. 删除 shell 脚本里的 `--time-scale`：
+   - `scripts/run_benchmark.sh:32, 70`: 删除 `--time-scale` 选项与传参。
+   - `scripts/run_experiments.sh:58`: 同上。
+
+4. 验证：
+   ```bash
+   python -m replayer --trace traces/w600_r0.0015_st30.jsonl \
+     --output /tmp/x.jsonl --endpoint http://localhost:9090 \
+     --max-inflight-sessions 64
+   # 不应再报 unrecognized arguments
+   ```
+
+5. 同步更新 `REPORT.md:430` 的 CLI 表格（删 `--time-scale`，保留 `--max-inflight-sessions`）。
+
+### 路线 B：彻底删掉这两个参数 + 删 shell 脚本
+
+如果不打算再跑高并发实验，则：
+1. 删 `scripts/run_benchmark.sh` 和 `scripts/run_experiments.sh`（与 D2 合并）。
+2. 修订 `REPORT.md:521, 541, 530` 中提到 `--max-inflight-sessions` 的全部段落，明确说"该参数已删除，对应实验留给后续工作"。
+
+**任选一条，但不能保留现状。**
+
+---
+
+<a id="b3"></a>
+## B3. PD-sep `--fire-and-forget` 路径损坏
+
+**严重度**: High（reachable-but-broken）。
+
+**定位**: `scripts/cache_aware_proxy.py:552-554, 570-573, 507-521`。
+
+**问题**:
+- `_handle_pd_sep` 在 `--fire-and-forget` 时 `asyncio.create_task(_send_prefill_async(...))` **不等 P 完成**。
+- 紧接 `:570-583` 立刻发起 D 端 decode，decode 携带 `remote_bootstrap_addr` + `remote_engine_id` + `transfer_id`。
+- 但 P 端此时可能尚未注册 `transfer_id`，Mooncake 拉取失败 → D 端 502。
+- 此外 `_send_prefill_async:507-521` 在异常分支只 `breakdown["prefill_error"] = True`，错误不会传递给 client。
+
+**改法**:
+
+如果按 [D1](#d1) 直接删，那这一条自动消失。
+否则按以下方式修：
+
+1. 在 `_send_prefill_async` 里加一个 `asyncio.Event`：
+   ```python
+   async def _send_prefill_async(p_inst, api, prefill_data, p_headers,
+                                  token_ids, input_length, breakdown,
+                                  ready: asyncio.Event):
+       try:
+           resp = await p_inst.client.post(api, json=prefill_data, headers=p_headers)
+           resp.raise_for_status()
+           await resp.aclose()
+           breakdown["t_prefill_done"] = _time.monotonic()
+           p_inst.record_prefix(token_ids)
+       except Exception as e:
+           breakdown["t_prefill_done"] = _time.monotonic()
+           breakdown["prefill_error"] = str(e)
+       finally:
+           p_inst.ongoing_tokens -= input_length
+           ready.set()
+   ```
+
+2. 在 `_handle_pd_sep` 里，发 decode 之前 `await ready.wait()`（带超时），保证 transfer_id 已注册：
+   ```python
+   ready = asyncio.Event()
+   asyncio.create_task(_send_prefill_async(..., ready=ready))
+   try:
+       await asyncio.wait_for(ready.wait(), timeout=PREFILL_TIMEOUT_S)
+   except asyncio.TimeoutError:
+       raise HTTPException(502, "Prefill not registered in time")
+   if "prefill_error" in breakdown:
+       raise HTTPException(502, breakdown["prefill_error"])
+   ```
+
+3. 这样语义其实就跟同步等待几乎一样了——更佳决策是按 [D1](#d1) 直接删。
+
+---
+
+<a id="b4"></a>
+## B4. 实现或移除 H4 cache-ratio gate
+
+**严重度**: High（design doc 与代码不一致 / fake feature）。
+
+**定位**: `scripts/cache_aware_proxy.py:288, 308`；`analysis/elastic_hypotheses.md`；`scripts/run_h4_cache_gate.sh`。
+
+**问题**:
+- `cache_ratio = cache_hit / max(input_length, 1)` 计算后**仅写入 breakdown**，没有任何分支根据它决策。
+- `analysis/elastic_hypotheses.md` 与 `run_h4_cache_gate.sh` 都假定"当 cache_ratio < 阈值时不 offload"；目前完全无效。
+
+**改法（推荐：实现）**:
+
+1. 在 `cache_aware_proxy.py` 顶部加常量与 CLI：
+   ```python
+   CACHE_GATE_RATIO = 0.3  # default; overridden by --cache-gate-ratio
+   ```
+   ```python
+   p.add_argument("--cache-gate-ratio", type=float, default=0.3,
+                  help="Min cache_hit/input ratio to allow offload "
+                       "(0.0 disables gate, 1.0 disables offload)")
+   ```
+   并在 `__main__` 里 `CACHE_GATE_RATIO = global_args.cache_gate_ratio`（参考 [D5](#d5)，最好不要用 module-level 赋值，直接读 args）。
+
+2. 在 `:312` 之前加 gate：
+   ```python
+   if cache_ratio < CACHE_GATE_RATIO:
+       offload_reason = "cache_gate_%.2f" % cache_ratio
+   elif current_offloads >= MAX_OFFLOAD_INFLIGHT:
+       offload_reason = "cap_reached_%d" % current_offloads
+   elif offload_cost < colocated_cost:
+       use_offload = True
+       offload_reason = "cost_model_%.1fvs%.1f" % (offload_cost, colocated_cost)
+   else:
+       offload_reason = "colocated_cheaper_%.1fvs%.1f" % (colocated_cost, offload_cost)
+   ```
+
+3. 把 `--cache-gate-ratio` 加到 `scripts/bench.sh` 与 `scripts/launch_phase1_ps.sh` 的 proxy 启动行（默认值 0.3，elastic 模式生效）。
+
+**或者（不实现）**: 把 `:288` 的 `cache_ratio` 计算与写入删除，并在 `analysis/elastic_hypotheses.md` 顶部加一句"H4 gate 设计未落地，结论待验证"。
+
+---
+
+<a id="b5"></a>
+## B5. `_percentile` off-by-one
+
+**严重度**: Medium（影响所有 summary 数据）。
+
+**定位**: `replayer/metrics.py:103-107`。
+
+**问题**:
+```python
+idx = round((len(sorted_vals) - 1) * pct)
+```
+对 len=100, pct=0.5 → `round(49.5) = 50`（Python banker's rounding 偶向偶）。
+对 len=2, pct=0.5 → `round(0.5) = 0`，但 `round(1.5) = 2` 等场景不稳定；银行家舍入让结果在偶数 idx 上偏倚。
+所有 p50 在偶数 sample 上偏向上中位。
+
+**改法**:
+
+替换为线性插值（与 numpy.percentile 默认一致）：
+
+```python
+def _percentile(sorted_vals: list[float], pct: float) -> float:
+    n = len(sorted_vals)
+    if n == 1:
+        return sorted_vals[0]
+    rank = pct * (n - 1)
+    lo = int(rank)
+    hi = min(lo + 1, n - 1)
+    frac = rank - lo
+    return sorted_vals[lo] * (1 - frac) + sorted_vals[hi] * frac
+```
+
+**验证**:
+```python
+# 单测：见 S1
+assert _percentile([1, 2, 3, 4], 0.5) == 2.5
+assert _percentile([1, 2], 0.5) == 1.5
+assert _percentile([1, 2, 3, 4, 5, 6, 7, 8, 9, 10], 0.9) == 9.1
+```
+
+---
+
+<a id="b6"></a>
+## B6. 统一 `bench.sh` 的模型路径
+
+**严重度**: Medium（新机器跑直接挂）。
+
+**定位**: `scripts/bench.sh:23`。
+
+**问题**:
+- `bench.sh:23`: `MODEL="${MODEL_PATH:-/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct}"`
+- 其它脚本 (`launch_vllm.sh`、`launch_elastic_p2p.sh`) 与 `TODO.md`：`$HOME/models/Qwen/Qwen3-Coder-30B-A3B-Instruct`。
+
+**改法**:
+
+把 `bench.sh:23` 的默认值改为：
+```bash
+MODEL="${MODEL_PATH:-$HOME/models/Qwen/Qwen3-Coder-30B-A3B-Instruct}"
+```
+
+并 `grep -rn "/home/admin/cpfs"` 检查整 repo 没有其它残留：
+```bash
+grep -rn "/home/admin/cpfs" /home/gahow/phd/agentic-kv
+```
+若有则一并替换为 `$HOME/models/...`。
+
+---
+
+<a id="m1"></a>
+## M1. `cached_blocks` 替换策略改为真正的 LRU
+
+**严重度**: Medium（router 估算 cache_hit 与真实 vLLM APC 长期偏差）。
+
+**定位**: `scripts/cache_aware_proxy.py:71-72`（`record_prefix`）。
+
+**问题**:
+```python
+if len(self.cached_blocks) > 200000:
+    self.cached_blocks = set(list(self.cached_blocks)[-100000:])
+```
+- `set` 迭代顺序在 CPython 不保证插入序，"取后 100k"等价于随机丢一半。
+- 这与 vLLM 内部 LRU 完全不一致，是 §3.6 提到的 24pp APC gap 的部分来源。
+
+**改法**:
+
+把 `cached_blocks: set[int]` 改成 `OrderedDict[int, None]` 充当 LRU：
+
+```python
+from collections import OrderedDict
+
+class InstanceState:
+    def __init__(self, ...):
+        ...
+        self.cached_blocks: OrderedDict[int, None] = OrderedDict()
+        self.cache_capacity = 200000  # blocks; tune with --cache-capacity-blocks
+
+    def estimate_cache_hit(self, token_ids):
+        if not token_ids or len(token_ids) < BLOCK_SIZE:
+            return 0
+        hit = 0
+        for i in range(0, len(token_ids) - BLOCK_SIZE + 1, BLOCK_SIZE):
+            bh = hash(tuple(token_ids[i:i + BLOCK_SIZE]))
+            if bh in self.cached_blocks:
+                self.cached_blocks.move_to_end(bh)  # LRU touch
+                hit += BLOCK_SIZE
+            else:
+                break
+        return hit
+
+    def record_prefix(self, token_ids):
+        if not token_ids:
+            return
+        for i in range(0, len(token_ids) - BLOCK_SIZE + 1, BLOCK_SIZE):
+            bh = hash(tuple(token_ids[i:i + BLOCK_SIZE]))
+            if bh in self.cached_blocks:
+                self.cached_blocks.move_to_end(bh)
+            else:
+                self.cached_blocks[bh] = None
+                if len(self.cached_blocks) > self.cache_capacity:
+                    self.cached_blocks.popitem(last=False)  # evict LRU
+```
+
+**进阶**: 容量应根据真实 KV cache 大小标定（vLLM 启动后 `total_blocks * block_size`），不要写死 200000。可以：
+- 加 `--cache-capacity-blocks` CLI（默认 200000）；
+- 或者从 vLLM `/metrics` 抓 `vllm:gpu_cache_usage_perc` 反推容量。
+
+---
+
+<a id="m2"></a>
+## M2. P 候选选择避开 `active_p_offloads`
+
+**严重度**: Medium。
+
+**定位**: `scripts/cache_aware_proxy.py:291-295`。
+
+**问题**:
+- 选 P 候选只按 `c.ongoing_tokens`，没有考虑某 instance 已经在为别人做 offload。
+- 配合 `MAX_OFFLOAD_INFLIGHT=4` 是 global cap，单 instance 可能扛多个 offload。
+
+**改法**:
+
+把 `:291-292` 的 key 加上 P-offload 罚项：
+
+```python
+def _p_pick_score(inst):
+    return (inst.ongoing_tokens
+            + inst.active_p_offloads * HEAVY_THRESHOLD)
+
+p_candidate = min((c for c in combined_instances if c is not best_inst),
+                  key=_p_pick_score)
+```
+
+并把 `MAX_OFFLOAD_INFLIGHT` 拆成 per-instance：
+
+```python
+if any(c.active_p_offloads >= MAX_OFFLOAD_PER_INSTANCE
+       for c in combined_instances):
+    # 全员上限，不 offload
+    ...
+elif p_candidate.active_p_offloads >= MAX_OFFLOAD_PER_INSTANCE:
+    offload_reason = "p_inst_cap_reached"
+```
+
+---
+
+<a id="m3"></a>
+## M3. 把 `MAX_OFFLOAD_INFLIGHT` 暴露为 CLI
+
+**严重度**: Low–Medium。
+
+**定位**: `cache_aware_proxy.py:32, 312`。
+
+**问题**: 模块常量 `MAX_OFFLOAD_INFLIGHT = 4`，未暴露 CLI；高并发实验时会成为隐性 bottleneck。
+
+**改法**:
+
+1. `parse_args` 里加：
+   ```python
+   p.add_argument("--max-offload-inflight", type=int, default=4,
+                  help="Global cap on concurrent P-role offloads")
+   ```
+
+2. 在 `_handle_combined` 里读 `global_args.max_offload_inflight` 而不是常量（与 [D5](#d5) 一致）。
+
+3. 同步 `bench.sh` / `launch_phase1_ps.sh`，elastic 模式可设大一点。
+
+---
+
+<a id="m4"></a>
+## M4. `session_affinity` 在 combined / pd-sep 之间命名空间隔离
+
+**严重度**: Low（当前不会同时跑两种模式，但属隐患）。
+
+**定位**: `cache_aware_proxy.py:158, 532`。
+
+**问题**: 全局 `session_affinity: dict[str, int]`；combined 模式 idx 指向 `combined_instances`，pd-sep 模式同 dict 又被 `pick_instance(prefill_instances, ...)` 写入并指向 `prefill_instances`。同一个 session_id 在两种模式下索引含义不同。
+
+**改法**:
+
+把 `session_affinity` 改成两个：
+
+```python
+session_affinity_combined: dict[str, int] = {}
+session_affinity_prefill: dict[str, int] = {}
+```
+
+`_handle_combined` 用前者，`_handle_pd_sep` 用后者。`pick_instance` 签名不变，只在调用方传不同 dict。
+
+---
+
+<a id="m5"></a>
+## M5. fallback 路径 client 断流时的资源泄漏
+
+**严重度**: Low–Medium（高并发下可能累积）。
+
+**定位**: `cache_aware_proxy.py:438-467`（`_handle_heavy_offload` fallback）；`:364-387`（`_handle_combined` 主路径）；`:585-598`（`_handle_pd_sep`）。
+
+**问题**:
+- StreamingResponse 返回后，若 client 在 generator 未被消费时断开，generator 不会进入 `try`，`finally` 不会触发。
+- 结果：`d_inst.ongoing_tokens` / `num_requests` / `pending_prefill_tokens` 永不释放，shadow state 与真实 load 越走越偏。
+- 长时间运行后 router 认为某些 instance 一直满载，路由失衡。
+
+**改法**:
+
+把"扣减"从 `finally` 换成 `BackgroundTasks`/`FastAPI` 的 lifecycle 不可靠，最稳妥是**在路由阶段就只做"加"，扣减在异步监听 client disconnect 的协程里做**。简化版改法：
+
+1. 包一层 `try/finally` 在调用 `StreamingResponse(generate(), ...)` 之前，并把状态扣减用 `request.is_disconnected()` 轮询或注册到 `BackgroundTask`。
+2. 或者更简单：在 `inst.ongoing_tokens += input_length` 的同时把"应在结束时扣减的值"塞进一个 dict（key=request_id），并在 `app` 层每 30s 扫一次 stale 请求（超过 `request_timeout * 2` 的）做兜底回收。
+
+**最小可行修复**：周期性 reconcile，在 `cache_aware_proxy.py` 里加一个后台 task：
+
+```python
+async def _reconcile_loop():
+    while True:
+        await asyncio.sleep(60)
+        for inst in combined_instances + prefill_instances + decode_instances:
+            # 简单 sanity: ongoing_tokens 永远 >= 0
+            if inst.ongoing_tokens < 0:
+                inst.ongoing_tokens = 0
+            if inst.num_requests < 0:
+                inst.num_requests = 0
+            # 进阶：与 vLLM /metrics 对账，详见 TODO.md item 6
+```
+
+并在 `lifespan` 启动该 task。这只是兜底，不解决根因；根因解决要走 TODO.md 第 6 条的 vLLM → Redis exact-state 路线。
+
+---
+
+<a id="m6"></a>
+## M6. `_send_prefill_async` 与同步路径的核算不一致
+
+**严重度**: Low（与 D1 一并解决）。
+
+**定位**: `cache_aware_proxy.py:507-521` vs `:556-568`。
+
+**问题**:
+- 同步路径在 finally 扣 `p_inst.ongoing_tokens`；
+- async 路径同样扣 `ongoing_tokens`，但 `pending_prefill_tokens` 在 PD-sep 路径中**两边都没维护**——表面一致，但与 combined 路径的语义不一致。
+
+**改法**: 看 [D1](#d1)。如果保留 fire-and-forget，加上 `breakdown` 的 ready event（[B3](#b3)）后，同时确保两路径核算字段对称。
+
+---
+
+<a id="d1"></a>
+## D1. 移除 `_send_prefill_async` 与 `--fire-and-forget`
+
+**严重度**: Cleanup。
+
+**定位**: `cache_aware_proxy.py:507-521`（function）、`:552-554`（caller）、`:634-635`（CLI flag）。
+
+**问题**:
+- grep 全 repo 所有 launch / bench / experiment 脚本，`--fire-and-forget` 0 处使用。
+- 配合 [B3](#b3)，这条 reachable 但 broken 的路径是 dead-on-arrival。
+
+**改法**:
+
+1. 删除 `_send_prefill_async` 整个函数。
+2. 删除 `_handle_pd_sep` 里 `if global_args.fire_and_forget: ... else:` 的分支，只保留同步 path。
+3. 删除 CLI 里的 `p.add_argument("--fire-and-forget", ...)`。
+4. `grep -rn "fire-and-forget\|fire_and_forget"` 确认无残留。
+
+---
+
+<a id="d2"></a>
+## D2. 删除/归档 `run_benchmark.sh` 与 `run_experiments.sh`
+
+**严重度**: Cleanup。
+
+**定位**: `scripts/run_benchmark.sh`、`scripts/run_experiments.sh`。
+
+**问题**: 与 [B2](#b2) 同源，两脚本仍传已删 CLI 参数；事实上不再可运行。
+
+**改法**:
+
+1. `mkdir -p scripts/legacy`
+2. `git mv scripts/run_benchmark.sh scripts/run_experiments.sh scripts/legacy/`
+3. 在 `scripts/legacy/README.md` 写一行："这些脚本对应早期 `--time-scale` / `--max-inflight-sessions` API，已归档，新实验请用 `scripts/bench.sh`。"
+4. 若选择 [B2](#b2) 路线 A 重新加回 `--max-inflight-sessions`，可顺便把 `run_benchmark.sh` 从 legacy 拉回并修参数。
+
+---
+
+<a id="d3"></a>
+## D3. 归档历史一次性 `analyze_*.py` / `compare_*.py`
+
+**严重度**: Cleanup（影响新人理解）。
+
+**定位**: `scripts/` 下约 20 个 `analyze_*.py` / `compare_*.py`。
+
+**问题**:
+- 大量脚本指向 `outputs/<exp>/...` 的旧实验路径（被 `.gitignore` 忽略，实际不存在）。
+- `compute_roofline.py:165` 硬编码 `traces/sampled_1000req_seed42.jsonl`（已不存在，详见 [D4](#d4)）。
+- 多个 `compare_*.py` 引用已删除实验目录。
+
+**改法**:
+
+1. 列一张表（在本文件下方"附录 A"或新建 `scripts/INVENTORY.md`），把每个 analyze/compare 脚本归类：
+   - **保留**: 有结构化用法、对当前 trace/output 仍可跑（如 `analyze_trace.py`、`analyze_breakdown.py`、`analyze_cache_hit.py`、`analyze_eviction.py`、`compare_results.py`）。
+   - **归档**: 一次性、特定实验 ID（如 `compare_ab_final.py`、`compare_balanced.py`、`compare_elastic_v4.py`、`compare_p2p.py`、`final_*.py`、`compare_aggregation.py`、`analyze_3way.py`、`analyze_h4_results.py`、`analyze_h5_rdma.py`、`profile_*.py`、`plot_gpu_timeline.py` 等）。
+
+2. `git mv` 归档类到 `scripts/legacy/`。
+
+3. 保留类的脚本：
+   - 顶部加 docstring，写明输入路径变量与示例命令。
+   - 凡是硬编码 `outputs/...` 路径的，全改成 `argparse` 参数。
+
+**最小行动**: 至少把以下"明显死"的归档：
+```
+scripts/legacy/
+├── compare_ab_final.py
+├── compare_adaptive.py
+├── compare_aggregation.py
+├── compare_balanced.py
+├── compare_elastic_v4.py
+├── compare_p2p.py
+├── final_all_comparison.py
+├── final_comparison.py
+├── final_gpu_comparison.py
+├── analyze_3way.py
+├── analyze_aggregation.py
+├── analyze_h4_results.py
+├── analyze_h5_rdma.py
+├── analyze_p2p_cache.py
+├── analyze_gpu_ab.py
+├── analyze_ablations.py
+├── plot_gpu_timeline.py
+├── profile_fnf.py
+├── profile_why_pdsep_loses.py
+├── ab_gpu_test.sh
+├── run_elastic_stability_test.sh
+├── run_h4_cache_gate.sh
+├── run_lmetric_ab.sh
+├── run_ps_ablation.sh
+├── run_ps_flexd.sh
+├── run_ps_remaining.sh
+└── run_v2_offload.sh
+```
+
+---
+
+<a id="d4"></a>
+## D4. 修正 `compute_roofline.py` 的硬编码 trace 路径
+
+**严重度**: Low。
+
+**定位**: `scripts/compute_roofline.py:165`。
+
+**问题**: 写死 `trace_path = "traces/sampled_1000req_seed42.jsonl"`，文件已不存在。
+
+**改法**:
+
+```python
+import argparse
+
+def main():
+    p = argparse.ArgumentParser()
+    p.add_argument("--trace", type=str,
+                   default="traces/w600_r0.0015_st30.jsonl",
+                   help="Trace JSONL path")
+    args = p.parse_args()
+    trace_path = args.trace
+    ...
+```
+
+---
+
+<a id="d5"></a>
+## D5. `HEAVY_THRESHOLD` / `OVERLOAD_FACTOR` 改读 args
+
+**严重度**: Low。
+
+**定位**: `cache_aware_proxy.py:30-34, 663-664, 88, 112`。
+
+**问题**:
+- 顶部 `HEAVY_THRESHOLD = 20000`，`__main__` 里 `HEAVY_THRESHOLD = global_args.heavy_threshold` 是给 module-level 名字赋值；
+- 函数体里 `_p_offload_penalty(inst)` 与 `pick_instance` 直接读 `HEAVY_THRESHOLD` 名字（globals），运行时正常生效；
+- 但若以后把 module 当库 import（例如加单测），`__main__` 块不执行，CLI 覆盖丢失。
+
+**改法**:
+
+把所有"运行时可调"常量挪到一个 `Settings` dataclass 里：
+
+```python
+from dataclasses import dataclass
+
+@dataclass
+class Settings:
+    heavy_threshold: int = 20000
+    overload_factor: float = 2.0
+    max_offload_inflight: int = 4
+    cache_gate_ratio: float = 0.3
+    prefill_throughput: float = 7000.0
+    rdma_overhead_s: float = 2.0
+    cache_capacity_blocks: int = 200000
+
+SETTINGS = Settings()
+```
+
+`parse_args` 后直接 `SETTINGS = Settings(**vars(args).filter(...))` 或逐字段赋值。函数体里改用 `SETTINGS.heavy_threshold` 等。
+
+---
+
+<a id="s1"></a>
+## S1. 给 `replayer/metrics.py` 与 cost-model 加单元测试
+
+**严重度**: Quality。
+
+**问题**: 整 repo 0 个测试。`_percentile`、`InstanceState.estimate_cache_hit`、`pick_instance`、cost-model 都该有最小覆盖。
+
+**改法**:
+
+1. 新建 `tests/` 目录，加 `tests/__init__.py`。
+2. `tests/test_metrics.py`：
+   ```python
+   from replayer.metrics import _percentile
+
+   def test_percentile_even():
+       assert _percentile([1, 2, 3, 4], 0.5) == 2.5
+
+   def test_percentile_odd():
+       assert _percentile([1, 2, 3, 4, 5], 0.5) == 3
+
+   def test_percentile_p99():
+       assert _percentile(list(range(1, 101)), 0.99) == 99.01
+   ```
+
+3. `tests/test_proxy_pick.py`：
+   ```python
+   import sys, pathlib
+   sys.path.insert(0, str(pathlib.Path(__file__).parent.parent / "scripts"))
+   from cache_aware_proxy import InstanceState, pick_instance, BLOCK_SIZE
+
+   def _new_inst(url="http://x"):
+       inst = InstanceState.__new__(InstanceState)
+       inst.url = url
+       inst.ongoing_tokens = 0
+       inst.pending_prefill_tokens = 0
+       inst.num_requests = 0
+       inst.active_p_offloads = 0
+       inst.cached_blocks = type(inst).__dict__.get(
+           "cached_blocks", set)()
+       return inst
+   # ...session affinity & overload tests
+   ```
+
+4. `pyproject.toml` 加 `[tool.pytest]` 段，跑 `pytest -q`。
+
+---
+
+<a id="s2"></a>
+## S2. 给 vLLM patch 加 import-time 校验
+
+**严重度**: Quality。
+
+**定位**: `patches/0001-fix-kv-transfer-abort-race.patch`。
+
+**问题**: 单 assert→warn 替换。未来升级 vLLM 时极易漏打 patch；当前没有运行时自检。
+
+**改法**:
+
+在 `scripts/cache_aware_proxy.py` 启动时（`lifespan` 开头）加：
+
+```python
+def _verify_vllm_patch():
+    """启动时自检：被 patch 的 scheduler 是否仍包含期望的 warn 路径。"""
+    import inspect
+    try:
+        from vllm.v1.core.sched.scheduler import Scheduler
+        src = inspect.getsource(Scheduler)
+        if "assert req_id in self.requests" in src:
+            print("WARNING: vLLM scheduler still has the unpatched assert; "
+                  "expect engine death on KV transfer abort race. "
+                  "Apply patches/0001-fix-kv-transfer-abort-race.patch.")
+    except Exception as e:
+        print(f"vLLM patch self-check skipped: {e}")
+```
+
+并在 `lifespan` 最开始调用。
+
+---
+
+<a id="s3"></a>
+## S3. REPORT.md 加 errata block
+
+**严重度**: Quality（避免读者引用过期结论）。
+
+**定位**: `REPORT.md` 顶部。
+
+**改法**:
+
+在 §1 后插入：
+
+```markdown
+## 0. Errata / 已废弃章节
+
+> 本报告为多次方法论修订后的累积版本，下列章节结论已被后续小节修订或推翻：
+>
+> - §3.1（PD-sep vs PD-combined 初版对比）：使用旧采样 + `--time-scale`，被 §3.6 推翻，**勿引用**。
+> - §3.5（elastic v3）：warm-vs-fresh 对比无效（baseline 实例未冷启动），**勿引用**。
+> - §X 中提到的 `--max-inflight-sessions 64+` 实验：CLI 已删除，对应实验需先按 FIXES.md B2 路线 A 恢复参数后再做。
+>
+> 当前**唯一权威的**结果章节为 §3.6 与 §3.7。
+```
+
+---
+
+## 验收清单
+
+修复完成后，按此清单逐项验证。
+
+- [ ] `grep -rn "_inst_cumulative_tokens" .` → 0 hits（B1）
+- [ ] `python -m replayer --help` 列表里**有**或**没有** `--max-inflight-sessions`（视 B2 路线选择，二者必须自洽）
+- [ ] `bash scripts/bench.sh ...` 在干净 repo 上能跑通至少 baseline 模式
+- [ ] `grep -rn "fire-and-forget\|fire_and_forget" scripts/` → 0 hits（D1 完成）
+- [ ] `grep -rn "/home/admin/cpfs" .` → 0 hits（B6）
+- [ ] `cache_aware_proxy.py` 中 `cache_ratio` 出现且**被某个分支引用**（B4 完成）
+- [ ] `pytest -q` 跑通新加的最小测试（S1）
+- [ ] `REPORT.md` 有 §0 Errata 段（S3）
+- [ ] 单跑 elastic 模式启动时打印 vLLM patch self-check 结果（S2）
+- [ ] `scripts/legacy/` 下能找到归档的脚本（D2、D3）
+- [ ] `_percentile([1,2,3,4], 0.5) == 2.5`（B5）
+
+## 修复顺序建议（按 PR 切分）
+
+1. **PR 1（不破坏行为，纯清理）**: B1、D1、D2、D3、D4、B6、S3
+2. **PR 2（修 bug）**: B5、M1、M5（轻量 reconcile）
+3. **PR 3（恢复实验能力）**: B2 路线 A（恢复 `--max-inflight-sessions`），同步 S1 加单测
+4. **PR 4（落地设计）**: B4（cache-ratio gate）、M2、M3、D5
+5. **PR 5（健壮性）**: M4、S2、剩余 M5 进阶版
+
+修完 PR 1–3 即可重新运行 REPORT 自己规定的 next-step 实验；PR 4–5 是 elastic 真正落地的前置。

REPORT.md

@@ -10,6 +10,26 @@
 
 For agentic LLM workloads (long input, short output, high KV cache reuse), is prefill-decode disaggregation beneficial? If full PD separation hurts (proven in §3), can **selective** disaggregation of only heavy requests improve serving latency while preserving KV cache locality?
 
+## 1.1 Errata / Superseded sections
+
+> This report has been revised several times as the methodology matured.
+> The sections below are kept for historical context but their numerical
+> conclusions have been **superseded** — do not cite them in isolation.
+>
+> - **§3.1 (initial PD-sep vs PD-combined)**: ran with the old random
+>   sampler + `--time-scale` compression + `--max-inflight-sessions 8`.
+>   Cross-session KV reuse dropped from 52% → 16%, and per-GPU concurrency
+>   was capped at 1 req/GPU. Superseded by §3.6.
+> - **Earlier "elastic v3" warm-vs-fresh runs**: baselines were not
+>   restarted between trials, leaving residual KV cache that inflated
+>   baseline TTFT ~2×. Superseded by the cold-start results in §3.6/§3.7.
+> - **Any reference to running `--max-inflight-sessions 64+`**: that flag
+>   was removed when replay moved to trace-driven dispatch. The next-step
+>   experiment requires restoring the flag first (see `FIXES.md` §B2
+>   route A) before any production-concurrency numbers can be produced.
+>
+> The authoritative results are in **§3.6 and §3.7**.
+
 ## 2. Experimental Setup
 
 ### 2.1 Hardware
@@ -42,10 +62,30 @@ For agentic LLM workloads (long input, short output, high KV cache reuse), is pr
 | Avg output tokens | 445 (p50=80) |
 | I/O ratio | 75.6× aggregate |
 | Prefill token share | 98% |
-| KV reuse (intra-session) | 91% of reusable blocks |
-| Theoretical max APC | 71% (infinite cache, single instance) |
+| KV reuse breakdown | 62% intra-session, 38% cross-session (token-level) |
+| Theoretical max APC | 67% (infinite cache, single instance, prefix-only) |
 
-**Sampled trace for benchmarks**: `traces/sampled_1000req_seed42.jsonl` (1000 requests, seed=42, preserving session structure). For 200-request ablations: replayer `--request-limit 200`.
+**Sampled trace for benchmarks**: `traces/w600_r0.0015_st30.jsonl` (1214 requests, 688 sessions, 70% multi-turn). Generated with window+thin sampling:
+
+```bash
+python scripts/sample_trace.py \
+    --input ~/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl \
+    --output traces/w600_r0.0015_st30.jsonl \
+    --sample-ratio 0.0015 --max-single-turn-ratio 0.3 \
+    --window-seconds 600 --seed 42
+```
+
+| Trace property | Value |
+|---------------|-------|
+| Sessions | 688 (70% multi-turn, avg 4.9 turns) |
+| Requests | 1214 (use `--request-limit 850` for daily, full for validation) |
+| Avg input tokens | 48,776 |
+| Trace span | 2912s (48.5 min); dense segment 0-990s (850 req) |
+| Peak QPS | 1.6 req/s (in dense segment) |
+| Hash block sharing | 48.3% (vs 52% full trace) |
+| Theoretical APC | 80% (full), 76% (first 850 req) |
+
+> **Sampling methodology (2026-05-23)**: Prior traces used random session sampling + `--time-scale` compression + `--max-inflight-sessions` semaphore, which (a) destroyed cross-session hash block sharing (52% → 16%), (b) artificially limited concurrency to 1 req/GPU, and (c) masked prefill-decode interference. The new approach uses contiguous time-window sampling with session thinning (`--max-single-turn-ratio 0.3`) to preserve KV reuse patterns, and trace-driven replay with no artificial concurrency limits.
 
 ### 2.4 Two Configurations Compared
 
@@ -119,9 +159,10 @@ python scripts/cache_aware_proxy.py \
 
 | Parameter | Value |
 |-----------|-------|
-| Requests | 200 (from sampled 1000-req trace, `--request-limit 200`) |
-| Time scale | 20× (compress 2h trace into ~6min) |
-| Max inflight sessions | 8 |
+| Trace | `traces/w600_r0.0015_st30.jsonl` (window+thin, 70% multi-turn) |
+| Daily iteration | `--request-limit 850` (~13 min, APC≈76%) |
+| Full validation | All 1214 requests (~48 min, APC≈80%) |
+| Replay mode | Trace-driven (no session limit, no time compression) |
 | Request timeout | 600s |
 | vLLM flags | `--enforce-eager --enable-prefix-caching --max-model-len 200000` |
 | GPU memory util | 0.9 |
@@ -139,31 +180,22 @@ python scripts/sample_trace.py \
     --output traces/sampled_1000req_seed42.jsonl \
     --target-requests 1000 --seed 42
 
-# Start GPU monitoring (in a separate terminal)
-bash scripts/gpu_monitor.sh > outputs/<tag>/gpu_util.csv &
+# Run benchmark (daily iteration)
+bash scripts/bench.sh --tag my_experiment --mode baseline --policy linear \
+    --trace traces/w600_r0.0015_st30.jsonl --requests 850
 
-# Run replayer against proxy
-python -m replayer \
-    --trace traces/sampled_1000req_seed42.jsonl \
-    --output outputs/<tag>/metrics.jsonl \
-    --endpoint http://localhost:9090 \
-    --time-scale 20 --max-inflight-sessions 8 \
-    --request-limit 200 -v
-
-# Collect proxy breakdown (elastic only)
-curl -s http://localhost:9090/breakdown > outputs/<tag>/breakdown.json
-
-# Collect APC from vLLM logs
-for i in $(seq 0 7); do
-    grep "Prefix cache hit rate\|External prefix cache hit rate" /tmp/<prefix>_$i.log | tail -2
-done
+# Run benchmark (full validation)
+bash scripts/bench.sh --tag my_experiment_full --mode baseline --policy linear \
+    --trace traces/w600_r0.0015_st30.jsonl
 ```
 
 ## 3. Results
 
-> **Errata (2026-05-22)**: The initial cross-machine A/B (dash0 baseline vs dash1 elastic) reported -44% E2E improvement. Post-hoc analysis revealed the dash0 baseline instances were **not freshly restarted** — residual KV cache from prior experiments caused 2× TTFT inflation. All results below use verified fresh-restart experiments on the same machine.
+> **Errata (2026-05-22)**: The initial cross-machine A/B (dash0 baseline vs dash1 elastic) reported -44% E2E improvement. Post-hoc analysis revealed the dash0 baseline instances were **not freshly restarted** — residual KV cache from prior experiments caused 2× TTFT inflation.
 
-### 3.1 Fair Comparison (all fresh-restart, same machine dash0, 200 req)
+> **Errata (2026-05-23)**: §3.1 results used artificial concurrency limits (`--max-inflight-sessions 8`, 1 req/GPU) and random session sampling that destroyed cross-session KV sharing (52% → 16%). See §3.6 for production-realistic results with corrected methodology.
+
+### 3.1 Legacy Comparison (artificial 1 req/GPU, 200 req)
 
 | Config | OK/N | TTFT p50 | TTFT p90 | TPOT p90 | E2E p50 |
 |--------|------|----------|----------|----------|---------|
@@ -201,14 +233,18 @@ Elastic's 3 extra errors are D-side KV pull failures: prefill succeeded on P, KV
 
 ### 3.4 Routing Policy: Linear vs LMetric (OSDI'26)
 
-LMetric (`score = P_tokens × BS`) vs linear (`score = ongoing_tokens - α·cache_hit`). Both fresh-restart, same trace.
+LMetric (`score = P_tokens × BS`, pure per-request, no session affinity) vs Linear (`score = ongoing_tokens - α·cache_hit`, session-sticky). Both fresh-restart, same trace.
+
+> **Errata (2026-05-23)**: Prior LMetric implementation incorrectly shared session-sticky logic with Linear. Fixed to pure per-request routing per OSDI'26 spec: `score = (pending_prefill + new_tokens) × num_requests`, no affinity, no overload override. Results below use corrected implementation.
 
 | Policy | TTFT p50 | TTFT p90 | TPOT p90 | E2E p50 | Delta E2E |
 |--------|----------|----------|----------|---------|-----------|
-| Linear | 1.086s | 9.432s | 0.077s | 5.423s | — |
-| LMetric | 1.099s | 9.392s | 0.073s | 5.205s | **-4.0%** |
+| Linear (session-sticky) | 1.073s | 9.347s | 0.073s | 5.119s | — |
+| LMetric (no affinity) | 1.081s | 9.408s | 0.072s | 5.102s | **-0.3%** |
 
-LMetric provides modest improvement through better load balancing. Routing policy headroom is limited for this workload.
+**Key finding**: LMetric without explicit session affinity matches Linear with session affinity on all metrics (< 2% difference). The cache-hit term in LMetric's scoring (`new_tokens = input - cache_hit`) creates **implicit soft affinity** — instances that already cached a session's prefix get lower P_tokens, naturally attracting subsequent turns. Explicit session-sticky routing is not required; cache-aware load balancing captures it automatically.
+
+APC distribution (LMetric, no affinity): inst_0=60.6%, inst_1=58.3%, inst_2=43.2%, inst_3=28.9%, inst_4=16.6%, inst_5=24.0%, inst_6=13.9%, inst_7=0.0%. Non-uniform but comparable aggregate to Linear's explicit affinity.
 
 ### 3.5 Errata: Why Prior Cross-Machine A/B Was Invalid
 
@@ -226,6 +262,140 @@ Delta:                   -45%          -44%    ← INVALID
 
 The elastic numbers on dash1 were genuinely fresh. The "improvement" was actually comparing fresh elastic against degraded baseline.
 
+### 3.6 Production-Realistic Baseline (trace-driven, corrected methodology)
+
+> Corrected sampling (window+thin, 70% multi-turn, block sharing 48%) and trace-driven replay (no session limit, no time compression). See §2.3 for trace details.
+
+**Linear policy, 912 requests (dense segment), peak QPS ≈ 1.6:**
+
+| Metric | Legacy (§3.1, 1 req/GPU) | **New (trace-driven)** | Delta |
+|--------|-------------------------|----------------------|-------|
+| TTFT mean | 1.07s | **4.54s** | +4.2× |
+| TTFT p50 | 1.08s | **0.94s** | -13% |
+| TTFT p90 | 9.38s | **14.12s** | +51% |
+| TPOT p50 | 0.038s | **0.070s** | **+84%** |
+| TPOT p90 | 0.073s | **0.175s** | **+139%** |
+| APC (mean) | ~44% | **67.5%** | **+23pp** |
+| Errors | 2/200 (1.0%) | 0/912 (0%) | better |
+| E2E p50 | 5.08s | 6.98s | +37% |
+
+**Key differences from legacy methodology:**
+
+1. **APC 67.5% vs 44%**: Window+thin sampling preserves cross-session block sharing (48% vs 16% in legacy random sampling), yielding production-realistic cache hit rates. Per-instance APC ranges 46–84%.
+
+2. **TPOT +139% at p90**: With trace-driven replay, multiple concurrent requests per GPU create **real prefill-decode interference**. The legacy 1 req/GPU setup showed TPOT p90=0.073s (no interference), but production-realistic load shows TPOT p90=0.175s. This validates that prefill-decode interference is a real problem at production concurrency.
+
+3. **TTFT p50 improved (-13%) but mean degraded (+4.2×)**: Higher APC means cached requests get very fast TTFT (p50=0.94s). But concurrent heavy prefills cause queuing for non-cached requests, inflating the mean and p90.
+
+4. **Per-instance APC imbalance (46–84%)**: Routing quality directly determines per-instance APC. The 38pp gap between worst and best instance suggests routing optimization is still the highest-leverage improvement.
+
+**Output**: `outputs/baseline_r0015_st30/` on dash0.
+
+### 3.7 Elastic PS vs Baseline (production-realistic trace)
+
+850 requests, `w600_r0.0015_st30.jsonl`, peak QPS≈1.6. Baseline on dash0, elastic on dash1.
+
+| Metric | Baseline | Elastic PS | Delta |
+|--------|----------|-----------|-------|
+| TTFT mean | 4.35s | 4.01s | -7.8% |
+| TTFT p50 | 0.94s | 0.93s | -1% |
+| TPOT p50 | 0.070 | 0.071 | +2% |
+| TPOT p90 | 0.162 | 0.157 | -3.1% |
+| E2E p50 | 6.38s | 6.44s | +0.9% |
+| APC mean | 60.7% | 59.9% | -0.8pp |
+| Errors | 0/850 | 4/832 | 4 ReadTimeout |
+
+**Elastic PS is near-neutral.** Root cause analysis:
+
+**Problem 1: Offload gate too restrictive** — only 17/118 HEAVY requests (14%) were offloaded. 75% of HEAVY requests had `cache_ratio=0%` (cold Turn 1), failing the `cache_ratio >= 0.3` gate. The gate was designed to avoid offloading cold requests (full prefill on P is slower than co-located), but this means 86% of HEAVY prefills still interfere with decode.
+
+**Problem 2: Offloaded requests are slower (+50.6%)** — HEAVY_OFFLOAD TTFT=19.94s vs HEAVY_COLO=13.25s. Breakdown:
+- Prefill on P: 14.72s (P also queued, no faster than co-located)
+- KV transfer + decode start on D: 5.71s (pure overhead)
+
+**Interference is real but unaddressed**: 89% of WARM/MEDIUM requests ran concurrently with 1+ HEAVY prefills (up to 60 concurrent). Elastic PS only offloaded 17/118 HEAVY requests — insufficient to reduce interference.
+
+**Conclusion**: The offload gate (`cache_ratio >= 0.3`) is correct in principle (cold offload IS slower), but leaves the core problem unsolved. Reducing prefill-decode interference requires either:
+1. Offloading ALL heavy prefills (accepting higher TTFT for offloaded requests in exchange for lower TPOT for all)
+2. Chunked prefill scheduling that yields to decode (vLLM-side optimization)
+3. Dedicated prefill GPUs (full PD separation) if KV memory wall can be solved
+
+**Output**: `outputs/eval_baseline_linear/` on dash0, `outputs/eval_elastic_linear/` on dash1.
+
+### 3.8 Direct KV Cache Migration (Bootstrap-Triggered PUSH)
+
+**Architecture**: D asks C's bootstrap server to PUSH cached KV blocks directly into D's GPU memory via Mooncake RDMA WRITE. C's vLLM scheduler is NOT involved (0 GPU compute on C). D then does local prefill for new tokens + decode.
+
+**Implementation details** (vLLM + Mooncake patches):
+
+1. **Hash table sync** (scheduler → worker → bootstrap): Each step, scheduler computes delta of `BlockPool.cached_block_hash_to_block` and syncs to worker's bootstrap server via `MooncakeConnectorMetadata.hash_table_updates`.
+
+2. **Token-based block lookup**: D sends `POST /push_blocks` with prompt `token_ids` + D's GPU addresses. C's bootstrap computes block hashes using `sha256` + `NONE_HASH` (same hash function as scheduler), matches against synced hash table.
+
+3. **RDMA PUSH**: C's bootstrap calls `TransferEngine.batch_transfer_sync_write` to push matched KV blocks from C's GPU into D's GPU. This uses the existing RDMA WRITE path (proven reliable), not RDMA READ (which fails on `batch_register_memory`'d GPU memory due to missing `IBV_ACCESS_REMOTE_READ` flags).
+
+4. **Cost model**: `offload when colocated_cost + interference > offload_cost`, where `interference = prefill_time × min(num_requests, 3) × 0.3`. Offload triggers when C has 1+ concurrent request.
+
+5. **Requirements**: `PYTHONHASHSEED` must be set (bench.sh sets `PYTHONHASHSEED=42` for elastic mode) to ensure deterministic `NONE_HASH` across scheduler/worker code paths.
+
+**Minimal test verification** (`scripts/test_direct_read.py`):
+
+| Metric | inst_0 (local cache) | inst_1 (RDMA push from inst_0) |
+|--------|---------------------|-------------------------------|
+| Turn 2 TTFT | 0.338s | **0.367s** |
+| Blocks transferred | — | **640/640 matched, push ret=0** |
+| External APC | 0% | **80%** |
+
+**Key bugs fixed during development**:
+- `NameError: field not imported` — missing dataclass import
+- Scheduler assertion crash (`assert RequestStatus.is_finished`) — partial remote prefill state mismatch
+- Hash mismatch 0/640 — `sha256` vs `sha256_cbor` (default hash algo is `sha256`, not `sha256_cbor`)
+- Hash mismatch 0/640 — `from X import NONE_HASH` creates stale value binding after `init_none_hash` reassigns the global; fixed with `import X; X.NONE_HASH`
+- RDMA READ ret=-1 — `batch_register_memory` only sets `IBV_ACCESS_REMOTE_WRITE`; switched to bootstrap-triggered PUSH
+- Cost model 0% trigger — removed stale `cache_gate_ratio` check; added interference penalty
+
+**Output**: `outputs/eval_direct_rdma_v*/` on dash0.
+
+### 3.9 Unified Routing (Final Design)
+
+Replaced two-phase routing (pick_instance → offload gate) with single `argmin(expected_latency)` per instance:
+
+```
+latency(D) = queue(D) + prefill(D) + transfer(D)
+  - Local cache: prefill = (input - local_hit) / throughput, transfer = 0
+  - PUSH from C: prefill = (input - push_hit) / throughput, transfer = 0.1s
+  - Cold:        prefill = input / throughput, transfer = 0
+```
+
+Session affinity as soft preference: use last instance if its cost ≤ 2× global best.
+
+Only 2 measured parameters: `prefill_throughput=7000 tok/s`, `rdma_overhead=0.1s`.
+
+**Results (eval_unified_v3, 850/850, 0 errors):**
+
+Baseline = `eval_baseline_linear` (plain vLLM, no Mooncake, linear policy, 850 req, same trace).
+
+| Metric | Baseline (plain) | **Unified v3 (kv_both)** | Delta |
+|--------|-----------------|-------------------------|-------|
+| TTFT mean | 4.348s | **3.277s** | **-24.6%** |
+| TTFT p50 | 0.945s | **0.793s** | **-16.1%** |
+| TTFT p90 | 12.468s | **8.472s** | **-32.1%** |
+| TTFT p99 | 48.149s | **41.587s** | **-13.6%** |
+| TPOT mean | 0.116s | 0.112s | -3.1% |
+| TPOT p50 | 0.071s | 0.077s | +8.9% |
+| TPOT p90 | 0.177s | 0.198s | +11.7% |
+| TPOT p99 | 1.018s | **0.816s** | **-19.9%** |
+| E2E mean | 19.10s | 19.81s | +3.7% |
+| E2E p50 | 6.443s | **5.599s** | **-13.1%** |
+| E2E p90 | 42.27s | 47.48s | +12.3% |
+| E2E p99 | 192.2s | 238.0s | +23.8% |
+
+**Routing**: 723 LOCAL + 116 PUSH_MIGRATE (13.8%). All 116 pushes had cache (avg 25k tokens) — no cold offloads. The unified cost model naturally avoids cold migration because `cold + RDMA > cold` (RDMA adds overhead without reducing prefill).
+
+**Tradeoff**: TTFT uniformly improves (p50 -16%, p90 -32%). TPOT mixed: p50/p90 worse (+9%/+12%), but p99 improves (-20%) — PUSH migration relieves the heaviest tail prefills. **E2E tail degrades significantly** (p90 +12%, p99 +24%): kv_both always-on overhead + PUSH transfer latency on migrated requests inflates E2E for long requests, offsetting the TTFT gain. The p50 benefit (-13%) comes from the majority of LOCAL requests getting faster prefill due to reduced queue contention.
+
+**Output**: `outputs/eval_unified_v3/` on dash0, baseline from `outputs/eval_baseline_linear/`.
+
 ## 4. System-Level Analysis
 
 ### 4.1 Elastic P2P Does Not Improve Single-Machine Performance
@@ -355,27 +525,118 @@ agentic-kv/
 | `scripts/gpu_monitor.sh` | Sample nvidia-smi to CSV | Pipe to `outputs/<tag>/gpu_util.csv` |
 | `scripts/launch_elastic_p2p.sh` | Launch all 8 kv_both instances + offload proxy | `HEAVY_THRESHOLD`, `MAX_OFFLOAD` env vars |
 
-## 8. Conclusions & Next Steps
+## 8. GPU Load Imbalance & Elastic Prefill Service Analysis
+
+### 8.1 Load Imbalance Characterization
+
+Session-sticky routing creates token load imbalance across instances. The severity depends on scale:
+
+| Scale | Imbalance | Top 5 sessions | Cause |
+|-------|-----------|----------------|-------|
+| 200 req (143 sessions) | **8.6×** tokens | 49% of all tokens | Small sample, few dominant sessions |
+| 1000 req (668 sessions) | **1.24×** tokens | 29% of all tokens | More sessions → natural averaging |
+
+At 1000 requests, the heaviest instance has 4.5M tokens vs lightest 3.6M (1.24×). Despite this, TPOT is uniform across all instances (0.070–0.077), confirming that prefill-decode interference is minimal at ≤1 session/GPU. The imbalance manifests in **TTFT only**: heaviest 2 instances TTFT p50 = 1.42s vs lightest 2 at 0.83s (1.7× gap).
+
+### 8.2 Session Accumulation Pattern
+
+Agentic workloads produce long-lived sessions with growing context:
+
+| Session | Turns | Total Tokens | Context Growth |
+|---------|-------|-------------|----------------|
+| 1569319 | 36 | 2.32M | 27k → 98k (+2.0k/turn) |
+| 1206593 | 36 | 2.31M | 15k → 106k (+2.6k/turn) |
+| 178176 | 25 | 1.93M | 36k → 95k (+2.5k/turn) |
+
+Top 5 sessions = 29% of all tokens. With session-sticky, these lock their instances, creating persistent load hotspots.
+
+### 8.3 Benchmark Concurrency vs Production Reality
+
+> **Critical caveat**: All prior experiments used `--max-inflight-sessions 8` (1 session/GPU). This is **10–15× below production concurrency** and masks the interference that elastic PS is designed to solve.
+
+| | Our Benchmark | Production Estimate |
+|--|---------------|---------------------|
+| Concurrent requests/GPU | 1–2 | **8–15** |
+| KV cache usage/GPU | 26–28% (single req) | **80–100%** |
+| Prefill-decode interference | Minimal | **Significant** |
+
+**KV cache capacity**: 281,888 tokens/GPU (25.8 GiB). A single 75k-token request consumes 27% of KV cache. At production concurrency (~15 req/GPU), KV cache is near-full, triggering eviction, cache misses, and prefill queuing — none of which appear in our 1-req/GPU benchmark.
+
+**Measured interference scaling**:
+
+| Concurrency | TPOT p90 | vs 8-session |
+|-------------|----------|-------------|
+| 8 sessions (1/GPU) | 0.075s | baseline |
+| 16 sessions (2/GPU) | 0.103s | **+38%** |
+| Production (~15/GPU) | *not tested* | *expected >>+45%* |
+
+### 8.4 Elastic PS: Two Capabilities Re-Evaluated
+
+**Capability 1: Reduce prefill-decode interference (lower TPOT)**
+
+At 1 req/GPU (our benchmark): no interference, no benefit. But this is an artifact of unrealistically low concurrency. At ≥2 req/GPU, chunked prefill interrupts decode steps, causing TPOT +38–45%. At production concurrency (~15/GPU), multiple HEAVY prefills sharing a GPU with decode requests would cause severe interference. Elastic PS's ability to isolate heavy prefills on separate GPUs directly addresses this.
+
+**Capability 2: Session migration for load balancing**
+
+Elastic PS enables mid-session migration: prefill on original instance (cache hit), KV transfer to a different instance for decode + future turns. This provides two benefits:
+
+1. **Break session lock-in**: Agentic sessions grow +2k tokens/turn over 30+ turns. With session-sticky, a 36-turn session (2.3M tokens total) locks its GPU, creating a hotspot. Elastic PS lets the session migrate to a less-loaded GPU while preserving cache on the original (PS does fast cached prefill, new GPU decodes).
+
+2. **Rebalance without cache loss**: Unlike breaking affinity (which destroys cache), elastic PS migration preserves the prefix cache on the original instance — the PS re-uses it for fast prefill, then transfers only the new KV to the destination.
+
+Simulation of migration strategies (1000 req, at current low concurrency):
+
+| Strategy | Imbalance | Migrations | KV Transfer Overhead |
+|----------|-----------|------------|---------------------|
+| No migration | 1.24× | 0 | 0s |
+| Every 10 turns | 1.21× | 10 | 15s |
+| Every 5 turns | 1.20× | 20 | 30s |
+
+At 1 req/GPU, migration benefit is marginal (imbalance is only 1.24×). At production concurrency where imbalance combines with KV cache pressure and interference, the benefit would be substantially larger.
+
+**Capability 3: Soft affinity from cache-hit scoring**
+
+The corrected LMetric experiment (§3.4) reveals that **explicit session affinity is unnecessary**. Cache-hit scoring (`new_tokens = input − cached`) creates implicit soft affinity — instances with cached prefixes score lower, naturally attracting subsequent turns. This matches hard session-sticky on all metrics (< 2% difference) while providing more flexible load balancing.
+
+### 8.5 Elastic PS Verdict
+
+| Aspect | At 1 req/GPU (tested) | At production load (expected) |
+|--------|----------------------|-------------------------------|
+| TPOT reduction | 0% (no interference) | **Significant** (interference scales with concurrency) |
+| Session migration | Marginal (1.24× → 1.20×) | **Larger benefit** (KV pressure + interference amplify imbalance) |
+| Cache preservation | N/A | **Key advantage** vs breaking affinity |
+
+**At our benchmark concurrency (1 req/GPU), elastic PS is not justified** — Mooncake overhead exceeds the non-existent interference benefit. **But our benchmark is 10–15× below production load.** The real question is whether elastic PS helps at production-realistic concurrency (64–128 concurrent sessions, 8–15 req/GPU), where:
+- Prefill-decode interference is measurable (already +38% TPOT at just 2/GPU)
+- KV cache pressure causes eviction and queue delays
+- Session accumulation creates compounding hotspots
+- Heavy prefills (50–100k tokens) block decode for seconds
+
+**Next step: run `--max-inflight-sessions 64` benchmark** to test elastic PS at production-realistic concurrency.
+
+## 9. Conclusions & Next Steps
 
 ### Established findings:
 1. Full PD separation is **net negative** for single-machine agentic workloads (KV cache memory wall)
-2. Cache-aware session-sticky routing is the **dominant optimization** (+24pp APC, -60% TTFT vs round-robin)
-3. **Elastic P2P offload does NOT improve single-machine performance** — Mooncake kv_both memory overhead (+11% TPOT, +37% E2E) outweighs prefill isolation benefit under moderate load (200 req)
-4. LMetric (OSDI'26) provides modest **E2E -4%** over linear routing; routing policy headroom is limited
-5. **Experimental methodology matters**: warm vs fresh instances cause 2× TTFT difference; all comparisons must use verified fresh restart
+2. Cache-aware routing is the **dominant optimization** (+24pp APC, -60% TTFT vs round-robin)
+3. **Explicit session affinity is unnecessary** — cache-hit scoring creates implicit soft affinity that matches hard session-sticky (< 2% difference)
+4. At low concurrency (1 req/GPU), elastic P2P offload adds overhead without benefit
+5. **Our benchmark concurrency is 10–15× below production**: `--max-inflight-sessions 8` yields 1 req/GPU, masking prefill-decode interference that appears at ≥2 req/GPU (+38% TPOT) and would dominate at production load (~15 req/GPU)
+6. **Experimental methodology matters**: warm vs fresh instances cause 2× TTFT difference
 
 ### Lessons learned:
-- Prior cross-machine A/B (commit `1e86285`) was invalid — warm baseline inflated by 2× due to residual KV cache state
+- Prior cross-machine A/B (commit `1e86285`) was invalid — warm baseline inflated by 2×
+- Prior LMetric implementation was invalid — incorrectly shared session-sticky logic with Linear
 - `kv_role=kv_both` has non-trivial always-on overhead even when P2P transfer is not used
 - Experiment isolation (kill all → verify GPU free → fresh start) is critical for reproducibility
+- **Benchmark concurrency must match production** — sub-production concurrency hides interference effects that drive system design decisions
 
-### Open problems:
-1. Elastic P2P may help under **sustained high load** (KV cache pressure makes co-located interference worse) — needs 1000-req experiment
-2. Mooncake kv_both memory overhead quantification and potential lazy initialization
-3. Multi-machine elastic (P on different node, no memory competition)
-4. Router state accuracy: proxy shadow state vs vLLM-internal exact state (TODO: vLLM → Redis → router)
-5. `scripts/bench.sh` standardized harness to prevent future warm-instance mistakes
+### Open problems (priority ordered):
+1. **Production-concurrency benchmark** (`--max-inflight-sessions 64+`): Validate whether prefill-decode interference at 8–15 req/GPU makes elastic PS net-positive
+2. **Multi-machine elastic**: P on a different node eliminates GPU memory competition — the main cost that makes single-machine elastic net negative
+3. **Layerwise KV transfer**: Mooncake's block-level transfer after full prefill is the bottleneck. Layerwise pipelining could reduce transfer latency by overlapping with computation
+4. **Router state accuracy**: proxy shadow state vs vLLM-internal exact state (TODO: vLLM → Redis → router)
 
 ---
 
-*Updated 2026-05-22. Prior elastic A/B results (commit `1e86285`) invalidated — see §3.5 errata.*
+*Updated 2026-05-23. LMetric corrected (§3.4 errata). GPU imbalance analysis added (§8). Benchmark concurrency gap identified — production-load experiments needed.*

docs/migration-policy-design.md

@@ -0,0 +1,78 @@
+# Migration Policy Design: Improving Load Balance in Elastic KV
+
+## Problem Statement
+
+With the unified cost model (v3), elastic routing achieves TTFT p90 -37% vs
+baseline on WARM/MEDIUM requests. However, **HEAVY turn>=2 requests with 99%
+cache hit still suffer TTFT 5-150s due to queuing contention** on overloaded
+instances.
+
+Root cause: the cost model combines cache benefit and queuing into a single
+scalar. When cache hit is 99%, the cost is dominated by queue estimation, but
+queue is inaccurately estimated via `(pending_prefill + decode_tokens) /
+throughput` — a token-based proxy that misses real contention (batch size).
+
+**Key data (v3, 850 requests, 8 instances):**
+- 391 turn>=2 HEAVY LOCAL requests were migration candidates
+- 298 (76%) had cache>80% — affinity held correctly
+- **38 of those 298 (13%) had TTFT>5s** despite 94-99% cache hit (queuing victims)
+- Only 8 offloads triggered total (2 real migrations, 6 useless turn-1 offloads)
+- Theoretical TTFT for turn2+ HEAVY: mean=0.81s (actual: 4.73s, **5.8x gap**)
+
+## Approach A: Contention-Aware Cost Model [ADOPTED]
+
+Replace `(pending_prefill + decode_tokens) / throughput` with
+`num_requests * decode_iteration_s + pending_prefill / throughput` as the
+queue estimation. `num_requests` (batch size) is the primary driver of
+decode iteration time and thus real contention.
+
+Add a migration discount for sessions with accumulated cache (turn >= 2),
+reflecting the long-term value of migrating a session off a loaded instance.
+
+### Parameters
+
+- `decode_iteration_s = 0.05` (per-request decode iteration cost on H20)
+- `migration_discount_cap = 5` (max turns to discount)
+
+### Results (vs baseline, 850 requests, 8×H20)
+
+| Metric           | Baseline | Approach A | Change  |
+|------------------|----------|------------|---------|
+| ALL TTFT mean    |    5.639 |      3.675 |   -35%  |
+| ALL TTFT p90     |   16.058 |      7.638 | **-52%**|
+| MEDIUM TTFT p90  |    4.412 |      1.681 | **-62%**|
+| HEAVY TTFT p90   |   23.780 |     15.929 |   -33%  |
+| ALL TPOT p90     |    0.105 |      0.075 |   -28%  |
+| ALL E2E p50      |    7.446 |      6.429 |   -14%  |
+| Errors           |        0 |          0 |     —   |
+
+## Approach B: Session-Level Lazy Migration [UNDER TUNING]
+
+Add a migration trigger **before** the cost model. When a request arrives for
+a session on an overloaded instance, force migration if:
+1. Instance busy: `num_requests > avg * migration_request_factor`
+2. Session has cache: `cache_ratio > 0.5`
+3. Request is HEAVY: `input_length >= heavy_threshold`
+4. Target meaningfully less loaded: `target.num_requests < source - 2`
+
+### Results (A+B combined, migration_request_factor=1.5)
+
+**0 migrations triggered** — Approach A's contention-aware routing already
+distributes load well enough that no instance reaches 1.5x average. The
+threshold needs to be lowered or the trigger redesigned.
+
+### Next steps
+
+- Lower `migration_request_factor` (e.g. 1.2 or 1.3)
+- Consider absolute threshold instead of relative (e.g. > avg + 3)
+- Or trigger based on recent TTFT rather than instantaneous num_requests
+
+## Evolution of Results
+
+| Version | Description | ALL TTFT p90 | HEAVY TTFT p90 | tok max/min |
+|---------|-------------|-------------|----------------|-------------|
+| Baseline | linear routing | 16.058 | 23.780 | 2.7x |
+| v2 (bug) | unified, queue=prefill only | 23.339 | 38.070 | 10.3x |
+| v3 | +decode in queue, +hard gate | 10.121 | 18.471 | 2.6x |
+| **A** | **+num_requests contention** | **7.638** | **15.929** | **3.5x** |
+| A+B | +session migration (1.5x) | 8.291 | 16.384 | 3.0x |

experiments/elastic_ps_eval.md

@@ -0,0 +1,120 @@
+# Elastic PS Evaluation Plan
+
+## Goal
+
+Compare **baseline (PD-combined)** vs **elastic PS (selective prefill offload)** under production-realistic trace on 8×H20.
+
+## Context
+
+The baseline (`baseline_r0015_st30`, 912 req) shows:
+- TPOT p90=0.175s (vs 0.073s at 1 req/GPU) — **prefill-decode interference is real**
+- APC=67.5% with per-instance range 46–84%
+- 58% of requests are HEAVY (≥20k), consuming 89% of input tokens
+
+Elastic PS offloads HEAVY prefills to a different GPU via Mooncake RDMA, isolating decode from prefill interference. Recent bug fixes:
+- D instance now accounted during prefill phase (prevents D overload)
+- MAX_OFFLOAD_INFLIGHT=4 cap prevents runaway offloads
+- D's proxy cache updated after decode (preserves session cache locality)
+
+## Machine
+
+dash0: 8×H20 96GB, NVLink, 4×CX7 200Gbps RDMA. SSH: `ssh dash0`.
+
+## Trace
+
+`traces/w600_r0.0015_st30.jsonl` on dash0 (1214 requests, 688 sessions, 70% multi-turn).
+Use `--requests 850` for ~13 min wall clock.
+
+## Experiments
+
+### Experiment 1: Baseline (Linear, PD-combined)
+
+```bash
+cd ~/agentic-kv && source .venv/bin/activate
+bash scripts/bench.sh \
+    --tag eval_baseline_linear \
+    --mode baseline --policy linear \
+    --trace traces/w600_r0.0015_st30.jsonl \
+    --requests 850
+```
+
+### Experiment 2: Elastic PS (Linear, kv_both + offload)
+
+```bash
+bash scripts/bench.sh \
+    --tag eval_elastic_linear \
+    --mode elastic --policy linear \
+    --trace traces/w600_r0.0015_st30.jsonl \
+    --requests 850
+```
+
+### Experiment 3: Baseline (LMetric, PD-combined)
+
+```bash
+bash scripts/bench.sh \
+    --tag eval_baseline_lmetric \
+    --mode baseline --policy lmetric \
+    --trace traces/w600_r0.0015_st30.jsonl \
+    --requests 850
+```
+
+### Experiment 4: Elastic PS (LMetric, kv_both + offload)
+
+```bash
+bash scripts/bench.sh \
+    --tag eval_elastic_lmetric \
+    --mode elastic --policy lmetric \
+    --trace traces/w600_r0.0015_st30.jsonl \
+    --requests 850
+```
+
+## What to Measure
+
+For each experiment, collect from `outputs/<tag>/`:
+1. `metrics.summary.json`: TTFT (mean/p50/p90), TPOT (mean/p50/p90), E2E, success rate
+2. `apc.txt`: per-instance prefix cache hit rate
+3. `breakdown.json`: per-request routing class (WARM/MEDIUM/HEAVY_COLO/HEAVY_OFFLOAD/HEAVY_COLO_FALLBACK)
+4. `stats.json`: per-instance load at end
+
+## Analysis
+
+After all 4 experiments, compare:
+
+```python
+import json
+
+def summarize(path):
+    s = json.load(open(path))
+    return {
+        "ok": "%d/%d" % (s["success_count"], s["request_count"]),
+        "ttft_mean": "%.2f" % s["ttft_stats_s"]["mean"],
+        "ttft_p50": "%.2f" % s["ttft_stats_s"]["p50"],
+        "ttft_p90": "%.2f" % s["ttft_stats_s"]["p90"],
+        "tpot_mean": "%.4f" % s["tpot_stats_s"]["mean"],
+        "tpot_p50": "%.4f" % s["tpot_stats_s"]["p50"],
+        "tpot_p90": "%.4f" % s["tpot_stats_s"]["p90"],
+        "e2e_p50": "%.2f" % s["latency_stats_s"]["p50"],
+    }
+
+for tag in ["eval_baseline_linear", "eval_elastic_linear",
+            "eval_baseline_lmetric", "eval_elastic_lmetric"]:
+    path = "outputs/%s/metrics.summary.json" % tag
+    print("%-30s %s" % (tag, summarize(path)))
+```
+
+Key questions:
+1. Does elastic PS reduce TPOT? (expect: yes, by isolating heavy prefills from decode)
+2. Does elastic PS hurt TTFT? (expect: some increase from RDMA overhead on offloaded requests)
+3. What's the net E2E impact? (TPOT improvement vs TTFT overhead)
+4. How many requests actually get offloaded? (check breakdown.json HEAVY_OFFLOAD count)
+5. Does the offload cap (MAX_OFFLOAD=4) get hit? (check breakdown for "cap_reached")
+6. Per-instance APC: does D maintain cache after migration? (compare APC spread)
+
+## Expected Results
+
+Based on analysis:
+- HEAVY requests: 58% of total, 89% of tokens
+- TPOT reduction potential: ~66% for WARM/MEDIUM (from 0.11 to 0.038)
+- RDMA overhead: ~1-15s per offloaded request (bimodal)
+- Net: TPOT should improve if offload successfully isolates prefill
+- Risk: Mooncake kv_both memory overhead may negate gains (was +11% TPOT in prior experiment at low concurrency)

pyproject.toml

@@ -14,3 +14,10 @@ dev = ["pytest"]
 [build-system]
 requires = ["hatchling"]
 build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["replayer"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-q"

replayer/__main__.py

@@ -17,10 +17,11 @@ def main() -> None:
     p.add_argument("--endpoint", type=str, required=True,
                    help="vLLM server URL (e.g. http://localhost:8000)")
     p.add_argument("--model", type=str, default="default", help="Model name for API")
-    p.add_argument("--time-scale", type=float, default=1.0,
-                   help="Time compression (>1 = faster)")
-    p.add_argument("--max-inflight-sessions", type=int, default=32)
-    p.add_argument("--concurrency-limit", type=int, default=256)
+    p.add_argument("--concurrency-limit", type=int, default=2000,
+                   help="Max concurrent HTTP requests (safety limit)")
+    p.add_argument("--max-inflight-sessions", type=int, default=None,
+                   help="Cap on concurrent sessions (None = unlimited; "
+                        "trace-driven dispatch otherwise)")
     p.add_argument("--request-timeout", type=float, default=600.0)
     p.add_argument("--request-limit", type=int, default=None,
                    help="Limit number of requests to replay")
@@ -37,11 +38,10 @@ def main() -> None:
         output_path=args.output,
         endpoint_url=args.endpoint.rstrip("/"),
         model_name=args.model,
-        time_scale=args.time_scale,
-        max_inflight_sessions=args.max_inflight_sessions,
         concurrency_limit=args.concurrency_limit,
         request_timeout_s=args.request_timeout,
         request_limit=args.request_limit,
+        max_inflight_sessions=args.max_inflight_sessions,
     )
 
     results = asyncio.run(replay_trace(config))

replayer/metrics.py

@@ -101,7 +101,11 @@ def _stats(values: list[float | None]) -> dict[str, float] | None:
 
 
 def _percentile(sorted_vals: list[float], pct: float) -> float:
-    if len(sorted_vals) == 1:
+    n = len(sorted_vals)
+    if n == 1:
         return sorted_vals[0]
-    idx = round((len(sorted_vals) - 1) * pct)
-    return sorted_vals[idx]
+    rank = pct * (n - 1)
+    lo = int(rank)
+    hi = min(lo + 1, n - 1)
+    frac = rank - lo
+    return sorted_vals[lo] * (1 - frac) + sorted_vals[hi] * frac

replayer/replay.py

@@ -1,16 +1,15 @@
 """Trace replayer — send requests to vLLM following trace timing.
 
-Supports both vLLM's /v1/completions (OpenAI-compatible) and /generate
-(SGLang-style) endpoints. Uses hash_ids from the trace to construct
-synthetic prompts that reproduce realistic prefix-cache hit patterns.
+Uses hash_ids from the trace to construct synthetic prompts that
+reproduce realistic prefix-cache hit patterns.
 
 Key behaviors:
+  - Trace-driven dispatch: each request is sent at its trace timestamp.
+    No artificial concurrency limits or time compression.
   - Per-session sequencing: turns within a session are sent in order,
     each waiting for the previous to complete before dispatching.
-  - Inter-session arrival: sessions start at their trace timestamps,
-    scaled by --time-scale.
-  - Concurrency control: --max-inflight-sessions caps concurrent sessions;
-    --concurrency-limit caps total in-flight requests.
+    If a turn completes after its successor's timestamp, the successor
+    fires immediately (no waiting for a past timestamp).
 """
 
 from __future__ import annotations
@@ -56,12 +55,11 @@ class ReplayConfig:
     trace_path: Path
     output_path: Path
     endpoint_url: str  # comma-separated for round-robin: "http://host:8000,http://host:8001"
-    time_scale: float = 1.0
-    max_inflight_sessions: int = 32
-    concurrency_limit: int = 256
+    concurrency_limit: int = 2000
     request_timeout_s: float = 600.0
     request_limit: int | None = None
     model_name: str = "default"
+    max_inflight_sessions: int | None = None  # cap on concurrent sessions; None = unlimited
 
 
 def _build_prompt_token_ids(req: TraceRequest) -> list[int]:
@@ -86,6 +84,12 @@ class _SessionState:
     metrics: list[RequestMetrics] = field(default_factory=list)
 
 
+@dataclass
+class _DispatchResult:
+    metric: RequestMetrics
+    output_token_ids: list[int]
+
+
 _endpoint_counter = 0
 
 
@@ -105,14 +109,17 @@ async def _dispatch_request(
     req: TraceRequest,
     prompt_token_ids: list[int],
     sem: asyncio.Semaphore,
-) -> RequestMetrics:
+) -> _DispatchResult:
     """Send one request via /v1/completions (streaming) and collect metrics."""
     endpoint = _pick_endpoint(config)
+    target_output_tokens = max(1, req.output_length)
     payload = {
         "model": config.model_name,
         "prompt": prompt_token_ids,
-        "max_tokens": max(1, req.output_length),
+        "max_tokens": target_output_tokens,
+        "min_tokens": target_output_tokens,
         "temperature": 0,
+        "return_token_ids": True,
         "stream": True,
         "stream_options": {"include_usage": True},
     }
@@ -124,6 +131,7 @@ async def _dispatch_request(
     finish_reason = None
     err = None
     token_times: list[float] = []
+    output_token_ids: list[int] = []
 
     req_headers = {"X-Session-Id": req.session_id}
 
@@ -148,14 +156,24 @@ async def _dispatch_request(
                     except json.JSONDecodeError:
                         continue
 
-                    now = time.perf_counter()
-                    if ttft_s is None:
-                        ttft_s = now - start
-
                     choices = chunk.get("choices", [])
                     if choices:
+                        now = time.perf_counter()
                         delta = choices[0].get("text", "")
-                        if delta:
+                        chunk_token_ids = choices[0].get("token_ids")
+                        if isinstance(chunk_token_ids, list):
+                            clean_ids = [
+                                int(t) for t in chunk_token_ids
+                                if isinstance(t, int)
+                            ]
+                            if clean_ids:
+                                if ttft_s is None:
+                                    ttft_s = now - start
+                                output_token_ids.extend(clean_ids)
+                                token_times.extend([now] * len(clean_ids))
+                        elif delta:
+                            if ttft_s is None:
+                                ttft_s = now - start
                             token_times.append(now)
                         fr = choices[0].get("finish_reason")
                         if fr:
@@ -170,8 +188,15 @@ async def _dispatch_request(
 
     end = time.perf_counter()
     e2e = end - start
-    if n_output == 0 and token_times:
+    if output_token_ids:
+        n_output = len(output_token_ids)
+    elif n_output == 0 and token_times:
         n_output = len(token_times)
+    if err is None and n_output != target_output_tokens:
+        err = (
+            "output_token_mismatch "
+            f"requested={target_output_tokens} actual={n_output}"
+        )
 
     tpot = 0.0
     if len(token_times) > 1:
@@ -179,26 +204,42 @@ async def _dispatch_request(
                        for i in range(len(token_times) - 1)]
         tpot = sum(inter_token) / len(inter_token)
 
-    return RequestMetrics(
-        request_id=req.request_id,
-        session_id=req.session_id,
-        turn_id=req.turn_id,
-        trace_timestamp_s=req.timestamp_s,
-        input_length=req.input_length,
-        output_length=req.output_length,
-        request_type=req.request_type,
-        effective_input_length=len(prompt_token_ids),
-        cached_tokens=cached_tokens,
-        latency_s=e2e,
-        ttft_s=ttft_s,
-        tpot_s=tpot,
-        actual_output_tokens=n_output,
-        requested_output_tokens=req.output_length,
-        finish_reason=finish_reason,
-        error=err,
+    return _DispatchResult(
+        metric=RequestMetrics(
+            request_id=req.request_id,
+            session_id=req.session_id,
+            turn_id=req.turn_id,
+            trace_timestamp_s=req.timestamp_s,
+            input_length=req.input_length,
+            output_length=req.output_length,
+            request_type=req.request_type,
+            effective_input_length=len(prompt_token_ids),
+            cached_tokens=cached_tokens,
+            latency_s=e2e,
+            ttft_s=ttft_s,
+            tpot_s=tpot,
+            actual_output_tokens=n_output,
+            requested_output_tokens=req.output_length,
+            finish_reason=finish_reason,
+            error=err,
+        ),
+        output_token_ids=output_token_ids,
     )
 
 
+def _apply_realized_prefix(
+    prompt_token_ids: list[int],
+    realized_context: list[int],
+) -> list[int]:
+    """Replace the reusable session prefix with engine-realized tokens."""
+    if not realized_context:
+        return prompt_token_ids
+    out = prompt_token_ids.copy()
+    n = min(len(out), len(realized_context))
+    out[:n] = realized_context[:n]
+    return out
+
+
 def _extract_cached_tokens(usage: dict) -> int:
     ct = 0
     details = usage.get("prompt_tokens_details")
@@ -214,34 +255,39 @@ async def _run_session(
     state: _SessionState,
     config: ReplayConfig,
     client: httpx.AsyncClient,
-    session_sem: asyncio.Semaphore,
     request_sem: asyncio.Semaphore,
     earliest_ts: float,
     sweep_start: float,
     sink: IncrementalMetricSink,
+    session_sem: asyncio.Semaphore | None = None,
 ) -> list[RequestMetrics]:
-    async with session_sem:
-        # Wait until this session's start time
-        offset = (state.turns[0].timestamp_s - earliest_ts) / config.time_scale
-        wait = offset - (time.perf_counter() - sweep_start)
-        if wait > 0:
-            await asyncio.sleep(wait)
-
+    if session_sem is not None:
+        await session_sem.acquire()
+    realized_context: list[int] = []
+    try:
         for req in state.turns:
-            # Intra-session: wait for turn's relative offset
-            if req != state.turns[0]:
-                target = (req.timestamp_s - state.turns[0].timestamp_s) / config.time_scale
-                elapsed = time.perf_counter() - sweep_start - offset
-                if elapsed < target:
-                    await asyncio.sleep(target - elapsed)
+            # Wait until this request's trace timestamp
+            target_wall = (req.timestamp_s - earliest_ts)
+            elapsed = time.perf_counter() - sweep_start
+            if elapsed < target_wall:
+                await asyncio.sleep(target_wall - elapsed)
 
-            token_ids = _build_prompt_token_ids(req)
-            metric = await _dispatch_request(
+            token_ids = _apply_realized_prefix(
+                _build_prompt_token_ids(req),
+                realized_context,
+            )
+            result = await _dispatch_request(
                 client=client, config=config, req=req,
                 prompt_token_ids=token_ids, sem=request_sem,
             )
+            metric = result.metric
             state.metrics.append(metric)
             await sink.append(metric)
+            if metric.error is None:
+                realized_context = token_ids + result.output_token_ids
+    finally:
+        if session_sem is not None:
+            session_sem.release()
 
     return state.metrics
 
@@ -283,16 +329,26 @@ async def replay_trace(config: ReplayConfig) -> list[RequestMetrics]:
 
     sessions = sorted(by_session.items(), key=lambda kv: kv[1][0].timestamp_s)
     earliest_ts = sessions[0][1][0].timestamp_s
+    latest_ts = max(r.timestamp_s for r in requests)
+    trace_span = latest_ts - earliest_ts
 
-    session_sem = asyncio.Semaphore(config.max_inflight_sessions)
     request_sem = asyncio.Semaphore(config.concurrency_limit)
+    session_sem = (
+        asyncio.Semaphore(config.max_inflight_sessions)
+        if config.max_inflight_sessions and config.max_inflight_sessions > 0
+        else None
+    )
 
     sink = IncrementalMetricSink(config.output_path)
 
     n_sessions = len(sessions)
     n_requests = len(requests)
-    logger.info("Replaying %d sessions (%d requests), time_scale=%.1f",
-                n_sessions, n_requests, config.time_scale)
+    qps = n_requests / trace_span if trace_span > 0 else 0
+    logger.info("Replaying %d sessions (%d requests) over %.0fs (%.2f req/s)",
+                n_sessions, n_requests, trace_span, qps)
+    if session_sem is not None:
+        logger.info("Session admission cap: %d concurrent sessions",
+                    config.max_inflight_sessions)
 
     pre_metrics = await _snapshot_prefix_cache_metrics(config.endpoint_url)
     sweep_start = time.perf_counter()
@@ -312,9 +368,10 @@ async def replay_trace(config: ReplayConfig) -> list[RequestMetrics]:
                 asyncio.create_task(_run_session(
                     state=_SessionState(session_id=sid, turns=turns),
                     config=config, client=client,
-                    session_sem=session_sem, request_sem=request_sem,
+                    request_sem=request_sem,
                     earliest_ts=earliest_ts, sweep_start=sweep_start,
                     sink=sink,
+                    session_sem=session_sem,
                 ))
                 for sid, turns in sessions
             ]

scripts/bench.sh

@@ -20,47 +20,56 @@ PROJECT_DIR="$(dirname "$SCRIPT_DIR")"
 VENV="${VENV_PATH:-$PROJECT_DIR/.venv/bin}"
 PYTHON="$VENV/python"
 VLLM="$VENV/vllm"
-MODEL="${MODEL_PATH:-/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct}"
-TRACE="$PROJECT_DIR/traces/sampled_1000req_seed42.jsonl"
+MODEL="${MODEL_PATH:-$HOME/models/Qwen/Qwen3-Coder-30B-A3B-Instruct}"
+TRACE="${TRACE:-$PROJECT_DIR/traces/w600_r0.0015_st30.jsonl}"
 
 # Defaults
 TAG=""
 MODE="baseline"     # baseline | elastic
-POLICY="linear"     # linear | lmetric
+POLICY="linear"     # linear | lmetric | unified
+POLICY_SET=false
 N_INSTANCES=8
 BASE_PORT=8000
 PROXY_PORT=9090
-REQUESTS=200
-TIME_SCALE=20
-MAX_SESSIONS=8
+REQUESTS=""         # empty = all requests in trace
 HEAVY_THRESHOLD=20000
 NO_OFFLOAD=false
 OVERLOAD_FACTOR_ARG=""
 MAX_BATCHED_TOKENS=""
+MAX_OFFLOAD_INFLIGHT=""
+CACHE_GATE_RATIO=""
+OFFLOAD_MODE=""
 
 # Parse args
 while [[ $# -gt 0 ]]; do
     case "$1" in
         --tag) TAG="$2"; shift 2 ;;
         --mode) MODE="$2"; shift 2 ;;
-        --policy) POLICY="$2"; shift 2 ;;
+        --policy) POLICY="$2"; POLICY_SET=true; shift 2 ;;
         --instances) N_INSTANCES="$2"; shift 2 ;;
         --requests) REQUESTS="$2"; shift 2 ;;
-        --time-scale) TIME_SCALE="$2"; shift 2 ;;
-        --sessions) MAX_SESSIONS="$2"; shift 2 ;;
+        --trace) TRACE="$2"; shift 2 ;;
         --heavy-threshold) HEAVY_THRESHOLD="$2"; shift 2 ;;
         --no-offload) NO_OFFLOAD=true; shift ;;
         --overload-factor) OVERLOAD_FACTOR_ARG="$2"; shift 2 ;;
         --max-batched-tokens) MAX_BATCHED_TOKENS="$2"; shift 2 ;;
+        --max-offload-inflight) MAX_OFFLOAD_INFLIGHT="$2"; shift 2 ;;
+        --cache-gate-ratio) CACHE_GATE_RATIO="$2"; shift 2 ;;
+        --offload-mode) OFFLOAD_MODE="$2"; shift 2 ;;
         *) echo "Unknown: $1"; exit 1 ;;
     esac
 done
 
 if [ -z "$TAG" ]; then
-    echo "Usage: bench.sh --tag NAME --mode {baseline|elastic} [--instances N] [--policy {linear|lmetric}] [--requests N]"
+    echo "Usage: bench.sh --tag NAME --mode {baseline|elastic} [--instances N] [--policy {linear|lmetric|unified}] [--requests N]"
+    echo "  Trace QPS is controlled by sample_trace.py --sample-ratio, not by bench.sh."
     exit 1
 fi
 
+if [ "$MODE" = "elastic" ] && [ "$POLICY_SET" = "false" ]; then
+    POLICY="unified"
+fi
+
 OUTDIR="$PROJECT_DIR/outputs/$TAG"
 if [ -d "$OUTDIR" ] && [ -f "$OUTDIR/metrics.jsonl" ]; then
     echo "[ERROR] Output directory $OUTDIR already exists with data. Use a different --tag."
@@ -76,9 +85,7 @@ cat > "$OUTDIR/config.json" << CONF
     "policy": "$POLICY",
     "model": "$MODEL",
     "n_instances": $N_INSTANCES,
-    "requests": $REQUESTS,
-    "time_scale": $TIME_SCALE,
-    "max_sessions": $MAX_SESSIONS,
+    "requests": "${REQUESTS:-all}",
     "heavy_threshold": $HEAVY_THRESHOLD,
     "no_offload": "$NO_OFFLOAD",
     "overload_factor": "${OVERLOAD_FACTOR_ARG:-2.0}",
@@ -91,12 +98,11 @@ CONF
 # ─── GPU Cleanup (verified) ────────────────────────────────────────────────
 
 cleanup_gpu() {
-    echo "[cleanup] Killing all vLLM/proxy processes..."
-    for p in $(ps aux | grep -E 'vllm serve|cache_aware_proxy' | grep -v grep | awk '{print $2}' 2>/dev/null); do
+    echo "[cleanup] Killing all vLLM/proxy/monitor processes..."
+    for p in $(ps aux | grep -E 'vllm serve|cache_aware_proxy|gpu_monitor' | grep -v grep | awk '{print $2}' 2>/dev/null); do
         kill -9 "$p" 2>/dev/null || true
     done
     sleep 3
-    # Kill any remaining GPU holders
     local gpu_pids
     gpu_pids=$(fuser /dev/nvidia* 2>/dev/null | tr ' ' '\n' | sort -u | grep -v '^$' || true)
     if [ -n "$gpu_pids" ]; then
@@ -104,7 +110,6 @@ cleanup_gpu() {
         echo "$gpu_pids" | xargs -r kill -9 2>/dev/null || true
         sleep 5
     fi
-    # Verify GPUs are free
     local used
     used=$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits 2>/dev/null | awk '{s+=$1} END{print s}')
     if [ "${used:-0}" -gt 100 ]; then
@@ -115,6 +120,9 @@ cleanup_gpu() {
     echo "[cleanup] All GPUs verified free."
 }
 
+trap 'echo "[bench.sh] Caught signal, cleaning up..."; cleanup_gpu; exit 1' INT TERM
+trap 'cleanup_gpu' EXIT
+
 # ─── Launch vLLM instances ─────────────────────────────────────────────────
 
 launch_instances() {
@@ -132,6 +140,7 @@ launch_instances() {
         local logfile="$OUTDIR/vllm_inst_${i}.log"
 
         if [ "$MODE" = "elastic" ]; then
+            PYTHONHASHSEED=42 \
             VLLM_MOONCAKE_BOOTSTRAP_PORT=$((8998 + i)) \
             MASTER_PORT=$master \
             CUDA_VISIBLE_DEVICES=$i \
@@ -210,6 +219,15 @@ launch_proxy() {
     if [ -n "$OVERLOAD_FACTOR_ARG" ]; then
         extra_args="$extra_args --overload-factor $OVERLOAD_FACTOR_ARG"
     fi
+    if [ -n "$MAX_OFFLOAD_INFLIGHT" ]; then
+        extra_args="$extra_args --max-offload-inflight $MAX_OFFLOAD_INFLIGHT"
+    fi
+    if [ -n "$CACHE_GATE_RATIO" ]; then
+        extra_args="$extra_args --cache-gate-ratio $CACHE_GATE_RATIO"
+    fi
+    if [ -n "$OFFLOAD_MODE" ]; then
+        extra_args="$extra_args --offload-mode $OFFLOAD_MODE"
+    fi
     if [ "$MODE" = "elastic" ]; then
         local bp_list=""
         for i in $(seq 0 $((N_INSTANCES - 1))); do
@@ -245,7 +263,13 @@ launch_proxy() {
 # ─── Run benchmark ─────────────────────────────────────────────────────────
 
 run_benchmark() {
-    echo "[bench] Running $REQUESTS requests (time_scale=$TIME_SCALE, sessions=$MAX_SESSIONS)..."
+    local request_args=""
+    if [ -n "$REQUESTS" ]; then
+        request_args="--request-limit $REQUESTS"
+        echo "[bench] Running $REQUESTS requests (trace-driven timing)..."
+    else
+        echo "[bench] Running all requests in trace (trace-driven timing)..."
+    fi
 
     # Start GPU monitor in background
     bash "$PROJECT_DIR/scripts/gpu_monitor.sh" "$OUTDIR/gpu_util.csv" 5 &
@@ -256,9 +280,7 @@ run_benchmark() {
         --output "$OUTDIR/metrics.jsonl" \
         --endpoint "http://localhost:$PROXY_PORT" \
         --model "$MODEL" \
-        --time-scale "$TIME_SCALE" \
-        --max-inflight-sessions "$MAX_SESSIONS" \
-        --request-limit "$REQUESTS" \
+        $request_args \
         -v 2>&1 | tee "$OUTDIR/replayer.log"
 
     # Stop GPU monitor
@@ -324,16 +346,17 @@ print('=' * 70)
 
 echo "================================================================"
 echo "  bench.sh: $TAG"
-echo "  mode=$MODE  policy=$POLICY  requests=$REQUESTS  overload_factor=${OVERLOAD_FACTOR_ARG:-2.0}  max_batched_tokens=${MAX_BATCHED_TOKENS:-default}"
+echo "  mode=$MODE  policy=$POLICY  requests=${REQUESTS:-all}  overload_factor=${OVERLOAD_FACTOR_ARG:-2.0}"
 echo "  $(date)"
 echo "================================================================"
 
+cd "$PROJECT_DIR"
 cleanup_gpu
 launch_instances
 launch_proxy
 run_benchmark
 collect_artifacts
 print_summary
-cleanup_gpu
+# cleanup_gpu runs automatically via EXIT trap
 
 echo "[done] $(date)"

scripts/compute_roofline.py

@@ -12,6 +12,7 @@ GPU: NVIDIA H20
   - Roofline ridge point: 148/4.0 = 37 FLOP/byte
 """
 
+import argparse
 import json
 import math
 
@@ -161,16 +162,25 @@ print("  PART 4: Agentic Workload Real Distribution")
 print("-" * 80)
 
 # Use actual trace data
-import os
-trace_path = "traces/sampled_1000req_seed42.jsonl"
-if os.path.exists(trace_path):
+_parser = argparse.ArgumentParser(description=__doc__)
+_parser.add_argument("--trace", type=str,
+                     default="traces/w600_r0.0015_st30.jsonl",
+                     help="Sampled trace JSONL for empirical workload roofline (Part 4)")
+_args, _ = _parser.parse_known_args()
+trace_path = _args.trace
+try:
+    _trace_fh = open(trace_path)
+except FileNotFoundError:
+    print(f"  (skipped: trace file not found: {trace_path})")
+    _trace_fh = None
+if _trace_fh is not None:
     BLOCK_SIZE = 512
     seen = set()
     compute_bound = 0
     memory_bound = 0
     total = 0
 
-    for line in open(trace_path):
+    for line in _trace_fh:
         d = json.loads(line)
         seq_len = d["input_length"]
         if seq_len < 1: continue
@@ -201,10 +211,12 @@ if os.path.exists(trace_path):
         else:
             memory_bound += 1
 
-    print(f"  With actual trace prefix cache pattern:")
-    print(f"    Compute-bound prefills: {compute_bound} ({compute_bound*100//total}%)")
-    print(f"    Memory-bound prefills:  {memory_bound} ({memory_bound*100//total}%)")
-    print(f"    (Decode is ALWAYS memory-bound at these seq lengths)")
-    print()
-    print(f"  Implication: {memory_bound*100//total}% of agentic prefills behave like decode")
-    print(f"  → PD separation treats them as 'compute-heavy' but they are actually memory-heavy")
+    _trace_fh.close()
+    if total > 0:
+        print(f"  With actual trace prefix cache pattern:")
+        print(f"    Compute-bound prefills: {compute_bound} ({compute_bound*100//total}%)")
+        print(f"    Memory-bound prefills:  {memory_bound} ({memory_bound*100//total}%)")
+        print(f"    (Decode is ALWAYS memory-bound at these seq lengths)")
+        print()
+        print(f"  Implication: {memory_bound*100//total}% of agentic prefills behave like decode")
+        print(f"  → PD separation treats them as 'compute-heavy' but they are actually memory-heavy")

scripts/deploy_vllm_patches.sh

@@ -0,0 +1,55 @@
+#!/bin/bash
+# Deploy modified vLLM Python files from third_party/ to site-packages.
+#
+# Usage: bash scripts/deploy_vllm_patches.sh [HOST]
+#   HOST: ssh alias (default: dash0). Use "local" for local deployment.
+#
+# This copies only the Python files we've modified — C extensions and
+# everything else come from the pip-installed vllm package.
+
+set -euo pipefail
+
+HOST="${1:-dash0}"
+PROJECT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+VLLM_SRC="$PROJECT_DIR/third_party/vllm/vllm"
+
+# Files modified relative to vllm/ package root
+PATCHED_FILES=(
+    "distributed/kv_transfer/kv_connector/v1/mooncake/mooncake_connector.py"
+    "distributed/kv_transfer/kv_connector/v1/mooncake/mooncake_utils.py"
+    "v1/core/sched/scheduler.py"
+)
+
+if [ "$HOST" = "local" ]; then
+    VENV_SITE=$("$PROJECT_DIR/.venv/bin/python" -c "import site; print(site.getsitepackages()[0])")
+    DST="$VENV_SITE/vllm"
+    echo "Deploying to local: $DST"
+    for f in "${PATCHED_FILES[@]}"; do
+        cp -v "$VLLM_SRC/$f" "$DST/$f"
+    done
+else
+    # Find site-packages on remote
+    VENV_SITE=$(ssh "$HOST" "~/agentic-kv/.venv/bin/python -c \"import site; print(site.getsitepackages()[0])\"")
+    DST="$VENV_SITE/vllm"
+    echo "Deploying to $HOST:$DST"
+    for f in "${PATCHED_FILES[@]}"; do
+        scp "$VLLM_SRC/$f" "$HOST:$DST/$f"
+    done
+fi
+
+echo "Deployed ${#PATCHED_FILES[@]} patched files."
+
+# Verify
+if [ "$HOST" = "local" ]; then
+    "$PROJECT_DIR/.venv/bin/python" -c "
+import vllm.distributed.kv_transfer.kv_connector.v1.mooncake.mooncake_utils as m
+print('mooncake_utils:', m.__file__)
+print('has estimate_hit:', hasattr(m.MooncakeBootstrapServer, 'estimate_hit'))
+"
+else
+    ssh "$HOST" "~/agentic-kv/.venv/bin/python -c \"
+import vllm.distributed.kv_transfer.kv_connector.v1.mooncake.mooncake_utils as m
+print('mooncake_utils:', m.__file__)
+print('has estimate_hit:', hasattr(m.MooncakeBootstrapServer, 'estimate_hit'))
+\""
+fi

scripts/launch_elastic_p2p.sh

@@ -90,6 +90,7 @@ $PYTHON "$PROJECT_DIR/scripts/cache_aware_proxy.py" \
     --combined $combined_args \
     --bootstrap-ports "$bootstrap_ports" \
     --offload \
+    --policy unified \
     --heavy-threshold $HEAVY_THRESHOLD \
     --port $PROXY_PORT &
 sleep 5

scripts/launch_phase1_ps.sh

@@ -4,10 +4,11 @@
 # 1 Prefill Service instance (GPU 7, port 8007, kv_both)
 set -euo pipefail
 
-cd /home/admin/cpfs/wjh/agentic-kv
+PROJECT_DIR="${PROJECT_DIR:-$HOME/phd/agentic-kv}"
+cd "$PROJECT_DIR"
 source .venv/bin/activate
 
-MODEL=/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct
+MODEL="${MODEL_PATH:-$HOME/models/Qwen/Qwen3-Coder-30B-A3B-Instruct}"
 OUTDIR=outputs/phase1_ps
 
 mkdir -p "$OUTDIR"

scripts/legacy/README.md

@@ -0,0 +1,20 @@
+# scripts/legacy
+
+One-shot scripts kept for historical reference. They were tied to specific
+experiments (cluster paths, deleted output dirs, removed CLI flags such as
+`--time-scale` / `--max-inflight-sessions`) and are no longer expected to
+run as-is.
+
+For new experiments use `scripts/bench.sh`. Pre-existing structured
+analyses still live in `scripts/` (e.g. `analyze_trace.py`,
+`analyze_breakdown.py`, `analyze_cache_hit.py`, `analyze_eviction.py`,
+`compare_results.py`, `compute_roofline.py`).
+
+If you need to revive a legacy script, expect to:
+
+- update hardcoded paths (cluster `/home/admin/cpfs/...`, deleted trace
+  files, missing `outputs/<exp>/...` directories);
+- adapt to the current replayer CLI (`--time-scale` and
+  `--max-inflight-sessions` were removed when methodology moved to
+  trace-driven dispatch);
+- re-verify the assumptions documented in `REPORT.md`.

scripts/legacy/run_lmetric_ab.sh

@@ -0,0 +1,56 @@
+#!/bin/bash
+# A/B comparison: linear (session-sticky) vs lmetric (OSDI'26, no affinity).
+# Wraps bench.sh for guaranteed fresh state between experiments.
+#
+# Usage:
+#   bash scripts/run_lmetric_ab.sh                    # defaults: 200 req
+#   bash scripts/run_lmetric_ab.sh --requests 1000    # override request count
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_DIR="$(dirname "$SCRIPT_DIR")"
+VENV="${VENV_PATH:-$PROJECT_DIR/.venv/bin}"
+TS=$(date +%Y%m%d_%H%M%S)
+
+echo "================================================================"
+echo "  A/B: Linear vs LMetric routing policy"
+echo "  $(date)"
+echo "================================================================"
+
+echo ""
+echo "=== Experiment 1/2: Linear (session-sticky) ==="
+bash "$SCRIPT_DIR/bench.sh" --tag "ab_linear_${TS}" --mode baseline --policy linear "$@"
+
+echo ""
+echo "=== Experiment 2/2: LMetric (no affinity) ==="
+bash "$SCRIPT_DIR/bench.sh" --tag "ab_lmetric_${TS}" --mode baseline --policy lmetric "$@"
+
+echo ""
+echo "================================================================"
+echo "  Results comparison"
+echo "================================================================"
+"$VENV/python" -c "
+import json
+
+def summarize(path):
+    rows = [json.loads(l) for l in open(path)]
+    ok = [r for r in rows if not r.get('error')]
+    p = lambda v,q: sorted(v)[min(int(q*len(v)),len(v)-1)] if v else 0
+    ttfts = [r['ttft_s'] for r in ok if r.get('ttft_s')]
+    tpots = [r['tpot_s'] for r in ok if r.get('tpot_s') and r['tpot_s']>0]
+    e2es = [r['latency_s'] for r in ok]
+    return len(ok), len(rows), p(ttfts,.5), p(ttfts,.9), p(tpots,.9), p(e2es,.5)
+
+lin = summarize('$PROJECT_DIR/outputs/ab_linear_${TS}/metrics.jsonl')
+lm = summarize('$PROJECT_DIR/outputs/ab_lmetric_${TS}/metrics.jsonl')
+
+fmt = '  %-20s OK=%3d/%3d  TTFT50=%7.3f  TTFT90=%7.3f  TPOT90=%6.4f  E2E50=%7.3f'
+print(fmt % ('Linear', *lin))
+print(fmt % ('LMetric', *lm))
+print()
+for name, i in [('TTFT50',2),('TTFT90',3),('TPOT90',4),('E2E50',5)]:
+    d = (lm[i] - lin[i]) / lin[i] * 100 if lin[i] else 0
+    print('  %s delta: %+.1f%%' % (name, d))
+"
+
+echo ""
+echo "Done at $(date)"

scripts/run_lmetric_ab.sh

@@ -1,149 +0,0 @@
-#!/bin/bash
-# A/B comparison: linear (current baseline) vs lmetric (OSDI'26) routing policy.
-# Both use same 8× TP=1 combined instances, fresh restart between experiments.
-set -euo pipefail
-
-PROJECT_DIR="/home/admin/cpfs/wjh/agentic-kv"
-VENV="$PROJECT_DIR/.venv/bin"
-VLLM="$VENV/vllm"
-PYTHON="$VENV/python"
-MODEL="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"
-TRACE="$PROJECT_DIR/traces/sampled_1000req_seed42.jsonl"
-
-N_INSTANCES=8
-BASE_PORT=8000
-PROXY_PORT=9090
-REQUEST_LIMIT=200
-TIME_SCALE=20
-MAX_SESSIONS=8
-
-cleanup() {
-    for p in $(ps aux | grep 'vllm serve' | grep -v grep | awk '{print $2}'); do kill -9 $p 2>/dev/null; done
-    for p in $(ps aux | grep 'cache_aware_proxy' | grep -v grep | awk '{print $2}'); do kill -9 $p 2>/dev/null; done
-    sleep 5
-    for p in $(fuser /dev/nvidia* 2>/dev/null | tr ' ' '\n' | sort -u); do kill -9 $p 2>/dev/null; done
-    sleep 10
-}
-
-start_instances() {
-    echo "  Starting $N_INSTANCES vLLM instances..."
-    for i in $(seq 0 $((N_INSTANCES - 1))); do
-        port=$((BASE_PORT + i))
-        MASTER_PORT=$((29500 + i)) CUDA_VISIBLE_DEVICES=$i \
-        $VLLM serve "$MODEL" \
-            --host 0.0.0.0 --port $port \
-            --tensor-parallel-size 1 \
-            --trust-remote-code --enable-prefix-caching --enforce-eager \
-            --dtype auto --gpu-memory-utilization 0.9 --max-model-len 200000 \
-            > /tmp/lmetric_ab_inst_$i.log 2>&1 &
-    done
-
-    echo "  Waiting for instances..."
-    for i in $(seq 0 $((N_INSTANCES - 1))); do
-        port=$((BASE_PORT + i))
-        timeout 600 bash -c "until curl -s localhost:$port/v1/models > /dev/null 2>&1; do sleep 5; done"
-        echo "    Instance $i (port $port) ready"
-    done
-}
-
-run_experiment() {
-    local policy=$1
-    local tag=$2
-    local outdir="$PROJECT_DIR/outputs/$tag"
-    mkdir -p "$outdir"
-
-    echo "  Starting proxy (policy=$policy)..."
-    $PYTHON "$PROJECT_DIR/scripts/cache_aware_proxy.py" \
-        --combined $(for i in $(seq 0 $((N_INSTANCES - 1))); do echo -n "http://127.0.0.1:$((BASE_PORT + i)) "; done) \
-        --policy "$policy" \
-        --port $PROXY_PORT > /tmp/lmetric_ab_proxy_${policy}.log 2>&1 &
-    PROXY_PID=$!
-    sleep 3
-
-    # Smoke test
-    result=$(curl -s -m 30 http://localhost:$PROXY_PORT/v1/completions \
-        -X POST -H "Content-Type: application/json" \
-        -d "{\"model\":\"$MODEL\",\"prompt\":[100,200,300],\"max_tokens\":3,\"temperature\":0}" 2>&1)
-    if ! echo "$result" | grep -q "choices"; then
-        echo "  ERROR: Smoke test failed: $result"
-        kill $PROXY_PID 2>/dev/null
-        return 1
-    fi
-    echo "  Smoke test passed"
-
-    # Start GPU monitor
-    bash "$PROJECT_DIR/scripts/gpu_monitor.sh" > "$outdir/gpu_util.csv" &
-    GPU_MON_PID=$!
-
-    # Run benchmark
-    echo "  Running benchmark (policy=$policy, $REQUEST_LIMIT requests)..."
-    $PYTHON -m replayer \
-        --trace "$TRACE" \
-        --output "$outdir/metrics.jsonl" \
-        --endpoint "http://localhost:$PROXY_PORT" \
-        --model "$MODEL" \
-        --time-scale $TIME_SCALE \
-        --max-inflight-sessions $MAX_SESSIONS \
-        --request-limit $REQUEST_LIMIT \
-        -v
-
-    # Save breakdown
-    curl -s http://localhost:$PROXY_PORT/breakdown > "$outdir/breakdown.json" 2>/dev/null
-    curl -s http://localhost:$PROXY_PORT/stats > "$outdir/stats.json" 2>/dev/null
-
-    # Collect APC from vLLM logs
-    echo "  Collecting APC..."
-    for i in $(seq 0 $((N_INSTANCES - 1))); do
-        pch=$(grep "Prefix cache hit rate" /tmp/lmetric_ab_inst_$i.log 2>/dev/null | tail -1 | grep -oP "Prefix cache hit rate: \K[0-9.]+" || echo "0")
-        echo "    inst_$i: prefix=$pch%"
-    done | tee "$outdir/apc.txt"
-
-    kill $GPU_MON_PID 2>/dev/null
-    kill $PROXY_PID 2>/dev/null
-    wait $PROXY_PID 2>/dev/null
-    echo "  Done: $(wc -l < "$outdir/metrics.jsonl") requests -> $outdir"
-}
-
-echo "================================================================"
-echo "  A/B: Linear vs LMetric routing policy"
-echo "  $(date)"
-echo "================================================================"
-
-# Experiment 1: Linear (current baseline)
-echo ""
-echo "=== Experiment 1: Linear policy ==="
-cleanup
-start_instances
-run_experiment "linear" "ab_linear"
-
-# Experiment 2: LMetric (OSDI'26)
-echo ""
-echo "=== Experiment 2: LMetric policy ==="
-cleanup
-start_instances
-run_experiment "lmetric" "ab_lmetric"
-
-# Compare
-echo ""
-echo "================================================================"
-echo "  Results comparison"
-echo "================================================================"
-$PYTHON -c "
-import json, statistics
-
-def summarize(path, label):
-    rows = [json.loads(l) for l in open(path)]
-    ok = [r for r in rows if not r.get('error')]
-    p = lambda v,q: v[min(int(q*len(v)),len(v)-1)] if v else 0
-    ttfts = sorted([r['ttft_s'] for r in ok if r.get('ttft_s')])
-    tpots = sorted([r['tpot_s'] for r in ok if r.get('tpot_s') and r['tpot_s']>0])
-    e2es = sorted([r['latency_s'] for r in ok])
-    print('%-20s OK=%3d/%3d  TTFT50=%.3f  TTFT90=%.3f  TPOT90=%.3f  E2E50=%.3f' % (
-        label, len(ok), len(rows), p(ttfts,.5), p(ttfts,.9), p(tpots,.9), p(e2es,.5)))
-
-summarize('$PROJECT_DIR/outputs/ab_linear/metrics.jsonl', 'Linear')
-summarize('$PROJECT_DIR/outputs/ab_lmetric/metrics.jsonl', 'LMetric')
-"
-
-echo ""
-echo "Done at $(date)"

scripts/sample_trace.py

@@ -2,22 +2,32 @@
 
 Preserves:
   - Complete session structure (all turns within a session kept together)
-  - Original arrival timing (inter-session and intra-session gaps)
-  - hash_ids for KV cache reuse patterns
+  - Original arrival timing (re-zeroed to t=0 but NOT compressed)
+  - KV cache reuse patterns (both intra-session AND cross-session sharing)
   - Request type distribution
 
-Sampling strategy:
-  1. Group requests by session (derived from parent_chat_id chains)
-  2. Randomly sample N sessions (or until target request count reached)
-  3. Re-zero timestamps so first event starts at t=0
-  4. Optionally compress time axis to increase load density
+Sampling strategy (--sample-ratio):
+  1. Take a contiguous time window from the trace (all sessions whose
+     first request falls within the window).  This preserves cross-session
+     hash block sharing because sessions that share system prompts appear
+     together in the same time region.
+  2. Within the window, randomly thin sessions by ratio to control QPS.
+  3. Re-zero timestamps so first event starts at t=0.
+
+The window is sized so that (window_sessions * thin_ratio) ≈ target count.
+Thin ratio is set high enough (≥0.5) to keep cross-session block sharing
+intact; the window width is narrowed to compensate.
 
 Usage:
+    # Sample for 8 GPUs from a ~500-GPU cluster
     python scripts/sample_trace.py \\
         --input ~/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl \\
         --output traces/sampled.jsonl \\
-        --target-requests 5000 \\
-        --seed 42
+        --sample-ratio 0.016 --seed 42
+
+    # Sample by request count (legacy, no sharing preservation)
+    python scripts/sample_trace.py \\
+        --input ... --output ... --target-requests 1000 --seed 42
 """
 
 from __future__ import annotations
@@ -26,7 +36,6 @@ import argparse
 import collections
 import json
 import random
-import sys
 from pathlib import Path
 
 
@@ -58,39 +67,127 @@ def load_raw_rows(path: Path) -> dict[str, list[dict]]:
 def sample_sessions(
     rows_by_session: dict[str, list[dict]],
     *,
-    target_requests: int,
+    sample_ratio: float | None = None,
+    target_requests: int | None = None,
+    max_single_turn_ratio: float | None = None,
+    window_seconds: float | None = None,
     seed: int,
-    strategy: str = "random",
 ) -> list[str]:
-    """Select sessions until target request count is reached."""
-    all_sids = list(rows_by_session.keys())
+    """Sample sessions preserving KV cache reuse."""
     rng = random.Random(seed)
 
-    if strategy == "random":
+    if sample_ratio is not None:
+        selected = _sample_window_then_thin(rows_by_session, sample_ratio,
+                                            window_seconds, rng)
+    elif target_requests is not None:
+        all_sids = list(rows_by_session.keys())
         rng.shuffle(all_sids)
-    elif strategy == "sequential":
-        pass  # keep file order
+        selected = []
+        total = 0
+        for sid in all_sids:
+            selected.append(sid)
+            total += len(rows_by_session[sid])
+            if total >= target_requests:
+                break
     else:
-        raise ValueError(f"Unknown strategy: {strategy}")
+        raise ValueError("Must specify --sample-ratio or --target-requests")
 
-    selected = []
-    total = 0
-    for sid in all_sids:
-        selected.append(sid)
-        total += len(rows_by_session[sid])
-        if total >= target_requests:
-            break
+    if max_single_turn_ratio is not None:
+        selected = _cap_single_turn(rows_by_session, selected,
+                                    max_single_turn_ratio, rng)
 
     return selected
 
 
+def _cap_single_turn(
+    rows_by_session: dict[str, list[dict]],
+    selected: list[str],
+    max_ratio: float,
+    rng: random.Random,
+) -> list[str]:
+    """Thin single-turn sessions so they are at most max_ratio of total sessions."""
+    multi = [s for s in selected if len(rows_by_session[s]) > 1]
+    single = [s for s in selected if len(rows_by_session[s]) == 1]
+
+    # max_ratio of TOTAL sessions should be single-turn
+    # n_single / (n_single + n_multi) <= max_ratio
+    # n_single <= max_ratio * n_multi / (1 - max_ratio)
+    max_single = int(max_ratio * len(multi) / (1 - max_ratio))
+    if len(single) <= max_single:
+        return selected
+
+    rng.shuffle(single)
+    return multi + single[:max_single]
+
+
+def _sample_window_then_thin(
+    rows_by_session: dict[str, list[dict]],
+    ratio: float,
+    window_seconds: float | None,
+    rng: random.Random,
+) -> list[str]:
+    """Window + thin sampling that preserves cross-session sharing.
+
+    1. Compute first-request timestamp for each session.
+    2. Pick a contiguous time window:
+       - If --window-seconds given: use that duration, thin by ratio within it.
+       - Otherwise: auto-size so window_sessions * thin_ratio ≈ target.
+    3. Keep all sessions whose first request falls within the window.
+    4. Randomly thin sessions within the window to hit target count.
+    """
+    session_starts: list[tuple[float, str]] = []
+    for sid, rows in rows_by_session.items():
+        t0 = min(float(r["timestamp"]) for r in rows)
+        session_starts.append((t0, sid))
+    session_starts.sort()
+
+    total_sessions = len(session_starts)
+    target_n = max(1, int(total_sessions * ratio))
+    trace_start = session_starts[0][0]
+    trace_end = session_starts[-1][0]
+    trace_duration = trace_end - trace_start
+
+    if window_seconds is not None:
+        # Fixed window: pick a random start, thin to hit target ratio
+        max_start_t = trace_end - window_seconds
+        if max_start_t <= trace_start:
+            win_start_t = trace_start
+        else:
+            win_start_t = trace_start + rng.random() * (max_start_t - trace_start)
+        win_end_t = win_start_t + window_seconds
+
+        window_sids = [sid for t, sid in session_starts
+                       if win_start_t <= t <= win_end_t]
+        # Thin to target
+        if len(window_sids) > target_n:
+            thin_ratio = target_n / len(window_sids)
+            window_sids = [s for s in window_sids if rng.random() < thin_ratio]
+        return window_sids
+
+    # Auto-size window
+    thin_ratio = min(1.0, max(0.5, ratio * 10))
+    window_sessions = min(int(target_n / thin_ratio), total_sessions)
+
+    max_start = total_sessions - window_sessions
+    window_start = rng.randint(0, max_start) if max_start > 0 else 0
+    window_sids = [sid for _, sid in
+                   session_starts[window_start:window_start + window_sessions]]
+
+    if thin_ratio < 1.0:
+        window_sids = [s for s in window_sids if rng.random() < thin_ratio]
+
+    if len(window_sids) > target_n * 1.2:
+        rng.shuffle(window_sids)
+        window_sids = window_sids[:int(target_n * 1.1)]
+
+    return window_sids
+
+
 def build_output(
     rows_by_session: dict[str, list[dict]],
     selected: list[str],
-    *,
-    time_scale: float = 1.0,
 ) -> list[dict]:
-    """Build output rows with re-zeroed timestamps."""
+    """Build output rows with re-zeroed timestamps (no time compression)."""
     out_rows = []
     for sid in selected:
         for row in rows_by_session[sid]:
@@ -103,10 +200,9 @@ def build_output(
     if not out_rows:
         return out_rows
 
-    # Re-zero: subtract earliest timestamp
     t0 = float(out_rows[0]["timestamp"])
     for row in out_rows:
-        row["timestamp"] = (float(row["timestamp"]) - t0) / time_scale
+        row["timestamp"] = float(row["timestamp"]) - t0
 
     return out_rows
 
@@ -125,20 +221,15 @@ def print_summary(
     output_lens = [r["output_length"] for r in out_rows]
 
     span_s = float(out_rows[-1]["timestamp"]) if out_rows else 0
-    session_starts = {}
-    for r in out_rows:
-        sid = r["session_id"]
-        ts = float(r["timestamp"])
-        if sid not in session_starts:
-            session_starts[sid] = ts
-    starts_sorted = sorted(session_starts.values())
-    deltas = [starts_sorted[i+1] - starts_sorted[i]
-              for i in range(len(starts_sorted) - 1)]
+    qps = n_requests / span_s if span_s > 0 else 0
 
-    # hash_ids overlap: count unique hash_ids across all requests
-    all_hashes = set()
+    # Hash block sharing
+    block_freq: dict[int, int] = collections.Counter()
     for r in out_rows:
-        all_hashes.update(r.get("hash_ids", []))
+        for h in r.get("hash_ids", []):
+            block_freq[h] += 1
+    total_blocks = len(block_freq)
+    shared_blocks = sum(1 for c in block_freq.values() if c > 1)
 
     print(f"Sampled: {n_sessions} sessions, {n_requests} requests")
     print(f"  Multi-turn sessions: {multi_turn} ({multi_turn/n_sessions*100:.1f}%)")
@@ -149,12 +240,8 @@ def print_summary(
     print(f"  Output length: min={min(output_lens)} max={max(output_lens)} "
           f"avg={sum(output_lens)/len(output_lens):.0f}")
     print(f"  Trace span: {span_s:.1f}s ({span_s/60:.1f} min)")
-    print(f"  Unique hash blocks: {len(all_hashes)}")
-    if deltas:
-        deltas.sort()
-        p = lambda q: deltas[min(int(q * len(deltas)), len(deltas) - 1)]
-        print(f"  Session arrival deltas (s): p10={p(0.1):.2f} p50={p(0.5):.2f} "
-              f"p90={p(0.9):.2f} max={max(deltas):.2f}")
+    print(f"  QPS: {qps:.2f} req/s")
+    print(f"  Hash blocks: {total_blocks} unique, {shared_blocks} shared ({shared_blocks*100/total_blocks:.1f}%)")
 
 
 def main() -> None:
@@ -164,15 +251,20 @@ def main() -> None:
                    help="Path to the full trace JSONL file")
     p.add_argument("--output", type=Path, required=True,
                    help="Path to write sampled trace JSONL")
-    p.add_argument("--target-requests", type=int, default=5000,
-                   help="Target number of requests (stops after session that crosses it)")
-    p.add_argument("--strategy", choices=["random", "sequential"], default="random",
-                   help="Session selection strategy")
-    p.add_argument("--time-scale", type=float, default=1.0,
-                   help="Compress time axis by this factor (>1 = faster arrival)")
+    p.add_argument("--sample-ratio", type=float, default=None,
+                   help="Fraction of sessions to sample (e.g. 0.016 for 8/500 GPU ratio)")
+    p.add_argument("--target-requests", type=int, default=None,
+                   help="Target number of requests (legacy, no sharing preservation)")
+    p.add_argument("--max-single-turn-ratio", type=float, default=None,
+                   help="Cap single-turn sessions to this fraction of total (e.g. 0.3)")
+    p.add_argument("--window-seconds", type=float, default=None,
+                   help="Time window duration in seconds (e.g. 600 for 10 min)")
     p.add_argument("--seed", type=int, default=42)
     args = p.parse_args()
 
+    if args.sample_ratio is None and args.target_requests is None:
+        p.error("Must specify --sample-ratio or --target-requests")
+
     print(f"Loading trace from {args.input} ...")
     rows_by_session = load_raw_rows(args.input)
     total_sessions = len(rows_by_session)
@@ -181,15 +273,14 @@ def main() -> None:
 
     selected = sample_sessions(
         rows_by_session,
+        sample_ratio=args.sample_ratio,
         target_requests=args.target_requests,
+        max_single_turn_ratio=args.max_single_turn_ratio,
+        window_seconds=args.window_seconds,
         seed=args.seed,
-        strategy=args.strategy,
     )
 
-    out_rows = build_output(
-        rows_by_session, selected,
-        time_scale=args.time_scale,
-    )
+    out_rows = build_output(rows_by_session, selected)
 
     print_summary(rows_by_session, selected, out_rows)
 

scripts/test_direct_read.py

@@ -0,0 +1,176 @@
+"""Minimal test: verify direct RDMA read hash matching.
+
+1. Send a multi-turn session to inst_0 (builds cache)
+2. Query inst_0's bootstrap /query_blocks with computed block hashes
+3. Check if hashes match (the core question)
+
+Usage:
+    # Start 2 elastic instances first, then:
+    python scripts/test_direct_read.py --port0 8000 --bp0 8998 --port1 8001 --bp1 8999
+"""
+
+import argparse
+import json
+import random
+import time
+
+import httpx
+
+BLOCK_SIZE = 512
+VOCAB_SIZE = 151936
+TOKEN_RANGE_START = 100
+TOKEN_RANGE_END = VOCAB_SIZE - 100
+
+
+def make_prompt(seed: int, n_blocks: int) -> list[int]:
+    """Deterministic prompt from seed, like the replayer does."""
+    rng = random.Random(seed)
+    return [rng.randint(TOKEN_RANGE_START, TOKEN_RANGE_END)
+            for _ in range(BLOCK_SIZE * n_blocks)]
+
+
+def main():
+    p = argparse.ArgumentParser()
+    p.add_argument("--port0", type=int, default=8000)
+    p.add_argument("--bp0", type=int, default=8998)
+    p.add_argument("--port1", type=int, default=8001)
+    p.add_argument("--bp1", type=int, default=8999)
+    p.add_argument("--model", type=str,
+                   default="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct")
+    args = p.parse_args()
+
+    client = httpx.Client(timeout=120)
+    base0 = f"http://127.0.0.1:{args.port0}"
+    base1 = f"http://127.0.0.1:{args.port1}"
+    bp0 = f"http://127.0.0.1:{args.bp0}"
+    bp1 = f"http://127.0.0.1:{args.bp1}"
+
+    # Step 1: Send request to inst_0 to build cache
+    prompt = make_prompt(seed=42, n_blocks=20)  # 10240 tokens
+    print(f"[1] Sending {len(prompt)} tokens to inst_0...")
+
+    resp = client.post(f"{base0}/v1/completions", json={
+        "model": args.model,
+        "prompt": prompt,
+        "max_tokens": 1,
+        "temperature": 0,
+    })
+    resp.raise_for_status()
+    print(f"    OK: {resp.json()['choices'][0]['text'][:20]}...")
+
+    # Wait for hash table sync (happens in scheduler step)
+    time.sleep(3)
+
+    # Step 2: Query inst_0's bootstrap for its hash table size
+    print(f"\n[2] Querying inst_0 bootstrap /query endpoint...")
+    resp = client.get(f"{bp0}/query")
+    resp.raise_for_status()
+    query_data = resp.json()
+    print(f"    Bootstrap has {len(query_data)} dp_rank entries")
+
+    # Step 3: Compute block hashes the way D would
+    # D's scheduler uses request.block_hashes which is computed by
+    # vLLM's block hasher. We can't easily replicate that here.
+    # Instead, let's send the SAME prompt to inst_1 with direct_read=True
+    # and see what happens.
+
+    # First, let's directly test the /query_blocks endpoint
+    # with some known hashes. We need to know what hashes inst_0 has.
+
+    # Try querying with dummy hashes to see the response format
+    print(f"\n[3] Testing /query_blocks with dummy hashes...")
+    resp = client.post(f"{bp0}/query_blocks", json={
+        "block_hashes": ["0000000000000000"],
+        "pin_token": "test-1",
+    })
+    resp.raise_for_status()
+    result = resp.json()
+    print(f"    Response: {json.dumps(result, indent=2)}")
+
+    # Unpin
+    client.post(f"{bp0}/unpin_blocks", json={"pin_token": "test-1"})
+
+    # Step 4: Send same prompt to inst_1 with do_remote_prefill + direct_read
+    # This triggers D's scheduler to compute block_hashes and the worker
+    # to query C's bootstrap
+    print(f"\n[4] Sending same prompt to inst_1 with direct_read...")
+
+    # Get inst_0's engine_id from bootstrap
+    engine_id = query_data.get("0", {}).get("engine_id", "")
+    print(f"    inst_0 engine_id: {engine_id}")
+
+    resp = client.post(f"{base1}/v1/completions", json={
+        "model": args.model,
+        "prompt": prompt,
+        "max_tokens": 1,
+        "temperature": 0,
+        "kv_transfer_params": {
+            "do_remote_decode": False,
+            "do_remote_prefill": True,
+            "direct_read": True,
+            "remote_bootstrap_addr": bp0,
+            "remote_engine_id": engine_id,
+            "transfer_id": "test-xfer-001",
+            "remote_num_tokens": len(prompt),
+        },
+    })
+    print(f"    Status: {resp.status_code}")
+    if resp.status_code == 200:
+        print(f"    Output: {resp.json()['choices'][0]['text'][:50]}...")
+    else:
+        print(f"    Error: {resp.text[:200]}")
+
+    # Step 5: Check logs for hash matching
+    print(f"\n[5] Check vLLM logs for direct_read activity:")
+    print(f"    grep 'direct_read\\|query_blocks\\|hash_table_sync\\|no cache hit' inst_*.log")
+
+    # Step 6: Send turn 2 (extended prompt) to verify prefix caching
+    prompt2 = prompt + make_prompt(seed=43, n_blocks=5)  # extend by 2560 tokens
+    print(f"\n[6] Sending turn 2 ({len(prompt2)} tokens) to inst_0...")
+    t0 = time.time()
+    resp = client.post(f"{base0}/v1/completions", json={
+        "model": args.model,
+        "prompt": prompt2,
+        "max_tokens": 1,
+        "temperature": 0,
+    })
+    resp.raise_for_status()
+    ttft = time.time() - t0
+    print(f"    TTFT: {ttft:.3f}s (should be fast if prefix cached)")
+
+    # Now send turn 2 to inst_1 with direct_read for turn 1's cache
+    print(f"\n[7] Sending turn 2 to inst_1 with direct_read (remote_num_tokens={len(prompt)})...")
+    t0 = time.time()
+    resp = client.post(f"{base1}/v1/completions", json={
+        "model": args.model,
+        "prompt": prompt2,
+        "max_tokens": 1,
+        "temperature": 0,
+        "kv_transfer_params": {
+            "do_remote_decode": False,
+            "do_remote_prefill": True,
+            "direct_read": True,
+            "remote_bootstrap_addr": bp0,
+            "remote_engine_id": engine_id,
+            "transfer_id": "test-xfer-002",
+            "remote_num_tokens": len(prompt),  # only first 10240 from remote
+        },
+    })
+    ttft1 = time.time() - t0
+    print(f"    Status: {resp.status_code}")
+    if resp.status_code == 200:
+        print(f"    TTFT: {ttft1:.3f}s")
+        print(f"    Output: {resp.json()['choices'][0]['text'][:50]}...")
+    else:
+        print(f"    Error: {resp.text[:200]}")
+
+    print(f"\n=== Summary ===")
+    print(f"Turn 1 on inst_0: OK")
+    print(f"Turn 2 on inst_0 (cached): TTFT={ttft:.3f}s")
+    print(f"Turn 2 on inst_1 (direct_read): TTFT={ttft1:.3f}s")
+    print(f"If direct_read works: inst_1 TTFT ≈ inst_0 TTFT (both have cache)")
+    print(f"If direct_read broken: inst_1 TTFT >> inst_0 TTFT (cold prefill)")
+
+
+if __name__ == "__main__":
+    main()

tests/test_metrics.py

@@ -0,0 +1,44 @@
+"""Tests for replayer.metrics percentile + summary helpers (B5)."""
+
+from __future__ import annotations
+
+import math
+
+from replayer.metrics import _percentile
+
+
+def test_percentile_single_value():
+    assert _percentile([42.0], 0.50) == 42.0
+    assert _percentile([42.0], 0.99) == 42.0
+
+
+def test_percentile_two_values_interpolates():
+    # For [0, 10] linear interpolation gives p50=5.0, p90=9.0.
+    assert math.isclose(_percentile([0.0, 10.0], 0.50), 5.0)
+    assert math.isclose(_percentile([0.0, 10.0], 0.90), 9.0)
+
+
+def test_percentile_endpoints():
+    vals = [1.0, 2.0, 3.0, 4.0, 5.0]
+    assert _percentile(vals, 0.0) == 1.0
+    assert _percentile(vals, 1.0) == 5.0
+
+
+def test_percentile_matches_numpy_linear_default():
+    # Independently computed using numpy's default linear interpolation;
+    # we hardcode the expectations so the test does not depend on numpy.
+    vals = [1.0, 2.0, 4.0, 8.0, 16.0, 32.0]
+    # rank for p50 = 0.5 * 5 = 2.5 -> 0.5 * 4 + 0.5 * 8 = 6.0
+    assert math.isclose(_percentile(vals, 0.50), 6.0)
+    # rank for p90 = 0.9 * 5 = 4.5 -> 0.5 * 16 + 0.5 * 32 = 24.0
+    assert math.isclose(_percentile(vals, 0.90), 24.0)
+    # rank for p99 = 0.99 * 5 = 4.95 -> 0.05 * 16 + 0.95 * 32 = 31.2
+    assert math.isclose(_percentile(vals, 0.99), 31.2)
+
+
+def test_percentile_no_off_by_one_at_boundary():
+    # Regression: previous round-based implementation returned the wrong
+    # element when rank fell exactly on an integer.
+    vals = [10.0, 20.0, 30.0]
+    # rank for p50 = 0.5 * 2 = 1.0 -> exactly element 1 -> 20.0
+    assert _percentile(vals, 0.50) == 20.0

tests/test_proxy_pick.py

@@ -0,0 +1,214 @@
+"""Minimal coverage for scripts/cache_aware_proxy pick_instance + cache LRU (S1)."""
+
+from __future__ import annotations
+
+import importlib.util
+import sys
+import types
+from pathlib import Path
+
+import pytest
+
+PROXY_PATH = Path(__file__).resolve().parent.parent / "scripts" / "cache_aware_proxy.py"
+
+
+def _install_stub_modules() -> None:
+    """Provide minimal stand-ins for fastapi/uvicorn/httpx so the proxy
+    module imports cleanly without the full server deps."""
+    if "uvicorn" not in sys.modules:
+        sys.modules["uvicorn"] = types.ModuleType("uvicorn")
+
+    if "fastapi" not in sys.modules:
+        fastapi_mod = types.ModuleType("fastapi")
+
+        class _FastAPI:
+            def __init__(self, *a, **kw):
+                self.state = types.SimpleNamespace()
+
+            def post(self, *a, **kw):
+                def deco(fn): return fn
+                return deco
+
+            def get(self, *a, **kw):
+                def deco(fn): return fn
+                return deco
+
+        class _HTTPException(Exception):
+            def __init__(self, status_code=500, detail=""):
+                self.status_code = status_code
+                self.detail = detail
+
+        class _Request:  # not actually instantiated by the routing tests
+            pass
+
+        fastapi_mod.FastAPI = _FastAPI
+        fastapi_mod.HTTPException = _HTTPException
+        fastapi_mod.Request = _Request
+        sys.modules["fastapi"] = fastapi_mod
+
+        responses_mod = types.ModuleType("fastapi.responses")
+
+        class _StreamingResponse:
+            def __init__(self, *a, **kw): pass
+
+        responses_mod.StreamingResponse = _StreamingResponse
+        sys.modules["fastapi.responses"] = responses_mod
+
+    if "httpx" not in sys.modules:
+        httpx_mod = types.ModuleType("httpx")
+
+        class _AsyncClient:
+            def __init__(self, *a, **kw): pass
+            async def aclose(self): pass
+
+        class _Limits:
+            def __init__(self, *a, **kw): pass
+
+        httpx_mod.AsyncClient = _AsyncClient
+        httpx_mod.Limits = _Limits
+        sys.modules["httpx"] = httpx_mod
+
+
+@pytest.fixture(scope="module")
+def proxy():
+    _install_stub_modules()
+    spec = importlib.util.spec_from_file_location("cache_aware_proxy", PROXY_PATH)
+    if spec is None or spec.loader is None:
+        pytest.skip(f"cannot load proxy module at {PROXY_PATH}")
+    mod = importlib.util.module_from_spec(spec)
+    sys.modules["cache_aware_proxy"] = mod
+    try:
+        spec.loader.exec_module(mod)
+    except ModuleNotFoundError as exc:
+        pytest.skip(f"proxy dependency missing: {exc}")
+    return mod
+
+
+def _make_inst(proxy, url: str, ongoing_tokens: int = 0,
+               active_p_offloads: int = 0):
+    inst = proxy.InstanceState(url)
+    inst.ongoing_tokens = ongoing_tokens
+    inst.active_p_offloads = active_p_offloads
+    return inst
+
+
+def test_record_prefix_evicts_oldest_block(proxy):
+    """LRU bound on cached_blocks must evict the oldest entry once full."""
+    inst = proxy.InstanceState("http://x")
+    saved = proxy.SETTINGS.cache_capacity_blocks
+    proxy.SETTINGS.cache_capacity_blocks = 2
+    try:
+        block_size = proxy.BLOCK_SIZE
+        # Three distinct one-block prefixes; first must be evicted.
+        prefix_a = [1] * block_size
+        prefix_b = [2] * block_size
+        prefix_c = [3] * block_size
+        inst.record_prefix(prefix_a)
+        inst.record_prefix(prefix_b)
+        inst.record_prefix(prefix_c)
+        assert len(inst.cached_blocks) == 2
+        # A should have been evicted.
+        assert inst.estimate_cache_hit(prefix_a) == 0
+        assert inst.estimate_cache_hit(prefix_b) == block_size
+        assert inst.estimate_cache_hit(prefix_c) == block_size
+    finally:
+        proxy.SETTINGS.cache_capacity_blocks = saved
+
+
+def test_estimate_cache_hit_touches_lru(proxy):
+    """A cache hit must move the block to the MRU position."""
+    inst = proxy.InstanceState("http://x")
+    saved = proxy.SETTINGS.cache_capacity_blocks
+    proxy.SETTINGS.cache_capacity_blocks = 2
+    try:
+        block_size = proxy.BLOCK_SIZE
+        a = [1] * block_size
+        b = [2] * block_size
+        c = [3] * block_size
+        inst.record_prefix(a)
+        inst.record_prefix(b)
+        # Touch A so it becomes MRU; B is now LRU.
+        assert inst.estimate_cache_hit(a) == block_size
+        # Insert C: B should be evicted, A should remain.
+        inst.record_prefix(c)
+        assert inst.estimate_cache_hit(a) == block_size
+        assert inst.estimate_cache_hit(b) == 0
+    finally:
+        proxy.SETTINGS.cache_capacity_blocks = saved
+
+
+def test_pick_instance_session_affinity_sticks(proxy):
+    insts = [_make_inst(proxy, "http://a"), _make_inst(proxy, "http://b")]
+    affinity = {"sess1": 1}
+    chosen, idx = proxy.pick_instance(insts, None, "sess1", 1000, affinity)
+    assert idx == 1 and chosen is insts[1]
+
+
+def test_pick_instance_session_affinity_breaks_on_overload(proxy):
+    """When the pinned instance is heavily overloaded, fallback to load-aware pick."""
+    insts = [
+        _make_inst(proxy, "http://a", ongoing_tokens=100),
+        _make_inst(proxy, "http://b", ongoing_tokens=1_000_000),
+        _make_inst(proxy, "http://c", ongoing_tokens=100),
+    ]
+    affinity = {"sess1": 1}
+    chosen, idx = proxy.pick_instance(insts, None, "sess1", 1000, affinity)
+    # avg ~333k; B at 1M is ~3x avg, well above OVERLOAD_FACTOR=2.0 -> fallback.
+    assert idx != 1
+    assert chosen is not insts[1]
+
+
+def test_pick_instance_p_offload_penalty_steers_away(proxy):
+    """Instances actively running offloaded HEAVY prefills get penalized."""
+    insts = [
+        _make_inst(proxy, "http://a", ongoing_tokens=0, active_p_offloads=2),
+        _make_inst(proxy, "http://b", ongoing_tokens=1000),
+    ]
+    chosen, idx = proxy.pick_instance(insts, None, None, 5000, {})
+    # B's 1000-token load is much smaller than A's 2 * HEAVY_THRESHOLD penalty.
+    assert idx == 1 and chosen is insts[1]
+
+
+def test_pick_instance_lmetric_picks_lowest_score(proxy):
+    insts = [_make_inst(proxy, "http://a"), _make_inst(proxy, "http://b")]
+    insts[0].pending_prefill_tokens = 0
+    insts[0].num_requests = 0
+    insts[1].pending_prefill_tokens = 5000
+    insts[1].num_requests = 4
+    chosen, idx = proxy.pick_instance_lmetric(insts, None, None, 1000, {})
+    # Empty instance has score = 1000 * 0 = 0; busy one has (5000+1000)*4.
+    assert idx == 0 and chosen is insts[0]
+
+
+def test_settings_has_runtime_knobs(proxy):
+    """D5/B4/M3: Settings dataclass exposes the previously-hardcoded knobs."""
+    s = proxy.SETTINGS
+    for field in (
+        "heavy_threshold",
+        "overload_factor",
+        "max_offload_inflight",
+        "cache_gate_ratio",
+        "prefill_throughput",
+        "rdma_overhead_s",
+        "cache_capacity_blocks",
+    ):
+        assert hasattr(s, field), f"SETTINGS missing {field}"
+    # Runtime mutability matters for tests + __main__ override.
+    saved = s.cache_gate_ratio
+    s.cache_gate_ratio = 0.55
+    assert proxy.SETTINGS.cache_gate_ratio == 0.55
+    s.cache_gate_ratio = saved
+
+
+def test_p_offload_penalty_uses_settings_heavy_threshold(proxy):
+    """M2: tweaking SETTINGS.heavy_threshold changes the P-offload penalty."""
+    inst = proxy.InstanceState("http://x")
+    inst.active_p_offloads = 3
+    saved = proxy.SETTINGS.heavy_threshold
+    try:
+        proxy.SETTINGS.heavy_threshold = 10000
+        assert proxy._p_offload_penalty(inst) == 30000
+        proxy.SETTINGS.heavy_threshold = 50000
+        assert proxy._p_offload_penalty(inst) == 150000
+    finally:
+        proxy.SETTINGS.heavy_threshold = saved

third_party/vllm/vllm/distributed/kv_transfer/kv_connector/v1/mooncake/mooncake_connector.py

@@ -5,7 +5,7 @@ import threading
 import time
 from collections import defaultdict
 from concurrent.futures import ThreadPoolExecutor
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 from enum import IntEnum
 from typing import TYPE_CHECKING, Any
 
@@ -65,6 +65,13 @@ TransferId = str  # KV transfer coordination ID (shared by P/D)
 
 logger = init_logger(__name__)
 
+# Module-level block pool for bootstrap server access (kv_both same-process only)
+_shared_block_pool = None
+
+def _set_shared_block_pool(bp):
+    global _shared_block_pool
+    _shared_block_pool = bp
+
 
 class MooncakeXferMetadata(
     msgspec.Struct,
@@ -108,6 +115,11 @@ class PullReqMeta:
     expire_time: float = float("inf")
     # Designed for one D pairing to multiple P
     pull_tasks_count: int = 0
+    # Direct RDMA read: D reads from C's GPU memory without C's scheduler
+    direct_read: bool = False
+    block_hashes: list[bytes] = field(default_factory=list)
+    prompt_token_ids: list[int] = field(default_factory=list)
+    remote_num_tokens: int = 0
 
 
 @dataclass
@@ -124,11 +136,13 @@ class SendBlockMeta:
 
 class MooncakeConnectorMetadata(KVConnectorMetadata):
     def __init__(self):
-        # Use (engine_id, dp_rank) to group reqs with same dp.
-        # See comments in MooncakeBootstrapServer.
         self.reqs_to_recv: dict[EngineId, dict[ReqId, PullReqMeta]] = defaultdict(dict)
         self.reqs_to_send: dict[ReqId, tuple[TransferId, list[int]]] = {}
         self.reqs_not_processed: set[TransferId] = set()
+        # Hash table sync: scheduler → worker (for direct RDMA read)
+        self.hash_table_updates: dict[str, int] = {}  # hex hash → block_id
+        self.hash_table_removals: set[str] = set()
+        self.token_hash_updates: dict[str, int] = {}  # str(hash(tokens)) → block_id
 
     def add_new_req(
         self,
@@ -136,16 +150,23 @@ class MooncakeConnectorMetadata(KVConnectorMetadata):
         local_block_ids: list[int],
         kv_transfer_params: dict[str, Any],
         load_remote_cache: bool = True,
+        block_hashes: list[bytes] | None = None,
+        prompt_token_ids: list[int] | None = None,
     ):
         transfer_id = kv_transfer_params["transfer_id"]
         if load_remote_cache:
             remote_engine_id = kv_transfer_params["remote_engine_id"]
+            remote_num = kv_transfer_params.get("remote_num_tokens", 0)
             self.reqs_to_recv[remote_engine_id][request_id] = PullReqMeta(
                 d_req_id=request_id,
                 local_block_ids=local_block_ids,
                 remote_engine_id=remote_engine_id,
                 remote_bootstrap_addr=kv_transfer_params["remote_bootstrap_addr"],
                 transfer_id=transfer_id,
+                direct_read=bool(kv_transfer_params.get("direct_read")),
+                block_hashes=block_hashes or [],
+                prompt_token_ids=prompt_token_ids or [],
+                remote_num_tokens=remote_num,
             )
         else:
             self.reqs_to_send[request_id] = (transfer_id, local_block_ids)
@@ -173,6 +194,12 @@ class MooncakeConnector(KVConnectorBase_V1):
             self.connector_scheduler = None
             self.connector_worker = MooncakeConnectorWorker(vllm_config, self.engine_id)
 
+    def set_block_pool(self, block_pool):
+        if self.connector_scheduler is not None:
+            self.connector_scheduler.set_block_pool(block_pool)
+        # Also store module-level for bootstrap server access (same process for kv_both TP=1)
+        _set_shared_block_pool(block_pool)
+
     ############################################################
     # Scheduler Side Methods
     ############################################################
@@ -222,6 +249,10 @@ class MooncakeConnector(KVConnectorBase_V1):
         assert self.connector_worker is not None
         return self.connector_worker.get_finished()
 
+    def get_block_ids_with_load_errors(self) -> set[int]:
+        assert self.connector_worker is not None
+        return self.connector_worker.get_block_ids_with_load_errors()
+
     def start_load_kv(self, forward_context: "ForwardContext", **kwargs) -> None:
         assert self.connector_worker is not None
         assert isinstance(self._connector_metadata, MooncakeConnectorMetadata)
@@ -250,6 +281,8 @@ class MooncakeConnectorScheduler:
 
     def __init__(self, vllm_config: VllmConfig, engine_id: str):
         self.vllm_config = vllm_config
+        self._block_pool = None
+        self._known_hash_keys: set = set()
 
         assert vllm_config.kv_transfer_config
         self.is_kv_producer: bool = (
@@ -260,14 +293,14 @@ class MooncakeConnectorScheduler:
         )
         logger.info("Initializing Mooncake Transfer Engine Scheduler %s", engine_id)
 
-        # Requests that need to start recv/send.
-        # New requests are added by update_state_after_alloc in
-        # the scheduler. Used to make metadata passed to Worker.
         self._reqs_need_recv: dict[ReqId, tuple[Request, list[int]]] = {}
         self._reqs_need_send: dict[ReqId, tuple[Request, list[int]]] = {}
-        # Reqs to remove from processed set because they're not to send after
-        # remote prefill or aborted.
         self._reqs_not_processed: set[TransferId] = set()
+        self._req_block_hashes: dict[ReqId, list[bytes]] = {}
+        self._req_token_ids: dict[ReqId, list[int]] = {}
+
+    def set_block_pool(self, block_pool):
+        self._block_pool = block_pool
 
     def get_num_new_matched_tokens(
         self, request: "Request", num_computed_tokens: int
@@ -299,14 +332,17 @@ class MooncakeConnectorScheduler:
             return 0, False
 
         if params.get("do_remote_prefill"):
-            # Remote prefill: get all prompt blocks from remote.
             assert not self.is_kv_producer
             token_ids = request.prompt_token_ids or []
-            count = len(token_ids) - num_computed_tokens
+            # Partial remote prefill: only pull remote_num_tokens from remote,
+            # compute the rest locally.  Falls back to full remote prefill
+            # when remote_num_tokens is not set.
+            remote_total = params.get("remote_num_tokens", len(token_ids))
+            remote_total = min(remote_total, len(token_ids))
+            count = max(0, remote_total - num_computed_tokens)
             if count > 0:
                 return count, True
 
-        # No remote prefill for this request.
         return 0, False
 
     def update_state_after_alloc(
@@ -330,21 +366,41 @@ class MooncakeConnectorScheduler:
                 p in params
                 for p in ("remote_engine_id", "remote_bootstrap_addr", "transfer_id")
             ):
-                # If remote_blocks and num_external_tokens = 0, we have
-                # a full prefix cache hit on the D worker. We need to call
-                # send_notif in _read_blocks to free the memory on the P.
-                local_block_ids = (
-                    blocks.get_unhashed_block_ids() if num_external_tokens > 0 else []
-                )
-                # Get unhashed blocks to pull from remote.
+                if num_external_tokens > 0:
+                    all_unhashed = blocks.get_unhashed_block_ids()
+                    # Partial remote prefill: only receive blocks for the
+                    # external portion, leave the rest for local compute.
+                    if params.get("remote_num_tokens") is not None:
+                        block_size = self.vllm_config.cache_config.block_size
+                        num_remote_blocks = (
+                            (num_external_tokens + block_size - 1) // block_size
+                        )
+                        local_block_ids = all_unhashed[:num_remote_blocks]
+                    else:
+                        local_block_ids = all_unhashed
+                else:
+                    local_block_ids = []
                 self._reqs_need_recv[request.request_id] = (request, local_block_ids)
+                if params.get("direct_read"):
+                    block_size = self.vllm_config.cache_config.block_size
+                    num_remote_blocks = (
+                        (num_external_tokens + block_size - 1) // block_size
+                    )
+                    if hasattr(request, "block_hashes"):
+                        self._req_block_hashes[request.request_id] = [
+                            bytes(h) for h in request.block_hashes[:num_remote_blocks]
+                        ]
+                    # Store prompt token_ids for token-based lookup on C
+                    if hasattr(request, "prompt_token_ids") and request.prompt_token_ids:
+                        self._req_token_ids[request.request_id] = list(
+                            request.prompt_token_ids[:num_remote_blocks * block_size]
+                        )
             else:
                 logger.warning(
                     "Got invalid KVTransferParams: %s. This "
                     "request will not utilize KVTransfer",
                     params,
                 )
-            # Only trigger 1 KV transfer per request.
             params["do_remote_prefill"] = False
 
         if params.get("do_remote_decode"):
@@ -361,7 +417,6 @@ class MooncakeConnectorScheduler:
     ) -> KVConnectorMetadata:
         meta = MooncakeConnectorMetadata()
 
-        # Loop through scheduled reqs and convert to PullReqMeta.
         if not self.is_kv_producer:
             for req_id, (req, block_ids) in self._reqs_need_recv.items():
                 assert req.kv_transfer_params is not None
@@ -369,9 +424,37 @@ class MooncakeConnectorScheduler:
                     request_id=req_id,
                     local_block_ids=block_ids,
                     kv_transfer_params=req.kv_transfer_params,
+                    block_hashes=self._req_block_hashes.pop(req_id, None),
+                    prompt_token_ids=self._req_token_ids.pop(req_id, None),
                 )
             self._reqs_need_recv.clear()
 
+        # Sync hash table to worker for direct RDMA read block lookups
+        if self._block_pool is not None:
+            cache = self._block_pool.cached_block_hash_to_block._cache
+            current_keys = set(cache.keys())
+            new_keys = current_keys - self._known_hash_keys
+            removed_keys = self._known_hash_keys - current_keys
+            if new_keys or removed_keys:
+                from vllm.v1.core.kv_cache_utils import get_block_hash
+                for k in new_keys:
+                    block = cache[k]
+                    if isinstance(block, dict):
+                        bid = next(iter(block.values())).block_id
+                    else:
+                        bid = block.block_id
+                    meta.hash_table_updates[get_block_hash(k).hex()] = bid
+                meta.hash_table_removals = {
+                    get_block_hash(k).hex() for k in removed_keys
+                }
+                self._known_hash_keys = current_keys.copy()
+                logger.info("hash_table_sync: +%d -%d (total known=%d)",
+                            len(new_keys), len(removed_keys), len(self._known_hash_keys))
+        else:
+            if not hasattr(self, '_bp_warned'):
+                logger.warning("_block_pool is None, hash table sync disabled")
+                self._bp_warned = True
+
         if not self.is_kv_consumer:
             for req_id, (req, block_ids) in self._reqs_need_send.items():
                 assert req.kv_transfer_params is not None
@@ -433,10 +516,13 @@ class MooncakeConnectorScheduler:
 
         # TODO: check whether block_ids actually ever be 0. If not we could
         # remove the conditional below
-        delay_free_blocks = len(block_ids) > 0
+        block_size = self.vllm_config.cache_config.block_size
+        prompt_blocks = (request.num_prompt_tokens + block_size - 1) // block_size
+        send_block_ids = block_ids[:prompt_blocks]
+        delay_free_blocks = len(send_block_ids) > 0
 
         if delay_free_blocks:
-            self._reqs_need_send[request.request_id] = (request, block_ids)
+            self._reqs_need_send[request.request_id] = (request, send_block_ids)
 
         return delay_free_blocks, None
 
@@ -541,6 +627,7 @@ class MooncakeConnectorWorker:
 
         self.finished_sending_reqs: set[ReqId] = set()
         self.finished_recving_reqs: set[ReqId] = set()
+        self.failed_recving_block_ids: set[int] = set()
 
         self.block_size = vllm_config.cache_config.block_size
         self.model_config = vllm_config.model_config
@@ -961,7 +1048,6 @@ class MooncakeConnectorWorker:
             "registered num_blocks=%d block_len=%d", self.num_blocks, self.block_len
         )
 
-        # No need to launch server for D node.
         if self.is_kv_consumer:
             return
 
@@ -969,13 +1055,27 @@ class MooncakeConnectorWorker:
         asyncio.run_coroutine_threadsafe(
             self._mooncake_sender_listener(ready_event), self.sender_loop
         )
-        ready_event.wait()  # Wait for listener ZMQ socket to be ready.
+        ready_event.wait()
+
+        if self.bootstrap_server is not None:
+            self.bootstrap_server.set_worker_kv_info(
+                self.kv_caches_base_addr, self.block_len,
+                self.block_size, self.hostname, self.rpc_port,
+                transfer_engine=self.engine,
+            )
+            if _shared_block_pool is not None:
+                self.bootstrap_server.set_block_pool(_shared_block_pool)
 
     async def fetch_finished_recving_reqs(self) -> set[ReqId]:
         finished_recving_reqs = self.finished_recving_reqs
         self.finished_recving_reqs = set()
         return finished_recving_reqs
 
+    def get_block_ids_with_load_errors(self) -> set[int]:
+        failed = self.failed_recving_block_ids
+        self.failed_recving_block_ids = set()
+        return failed
+
     async def fetch_finished_sending_reqs(self) -> set[ReqId]:
         finished_sending_reqs = self.finished_sending_reqs
         self.finished_sending_reqs = set()
@@ -1089,6 +1189,10 @@ class MooncakeConnectorWorker:
             logger.debug("ZMQ context terminated, exiting Mooncake receiver thread.")
         except Exception as e:
             logger.error("MooncakeXferMetadata transfer failed for %s: %s", req_ids, e)
+            for req_id in req_ids:
+                pull_meta = pull_metas[req_id]
+                self.failed_recving_block_ids.update(pull_meta.local_block_ids)
+                self.finished_recving_reqs.add(pull_meta.d_req_id)
             return
 
     def process_pulling_result(
@@ -1114,6 +1218,12 @@ class MooncakeConnectorWorker:
                 response.err_reqs,
                 response.err_msg,
             )
+            for req_id in response.err_reqs:
+                pull_meta = pull_metas.get(req_id)
+                if pull_meta is None:
+                    continue
+                self.failed_recving_block_ids.update(pull_meta.local_block_ids)
+                self.finished_recving_reqs.add(pull_meta.d_req_id)
 
     async def _connect_to_prefiller_bootstrap(self, remote_bootstrap_addr: str):
         url = remote_bootstrap_addr + "/query"
@@ -1187,6 +1297,63 @@ class MooncakeConnectorWorker:
 
         self.receive_kv(remote_engine_id, pull_metas)
 
+    async def _start_direct_read(
+        self, reqs_by_engine: dict[EngineId, dict[ReqId, PullReqMeta]]
+    ):
+        """Direct RDMA read: D reads cached KV blocks from C's GPU memory
+        without involving C's scheduler.
+        """
+        for _engine_id, pull_metas in reqs_by_engine.items():
+            for req_id, pm in pull_metas.items():
+                asyncio.create_task(
+                    self._direct_read_single(req_id, pm)
+                )
+
+    async def _direct_read_single(self, req_id: ReqId, pm: PullReqMeta):
+        """Bootstrap-triggered PUSH: D asks C's bootstrap to push matched blocks.
+
+        C's bootstrap looks up cached blocks by token_ids, then uses C's
+        TransferEngine to RDMA WRITE (push) them directly into D's GPU memory.
+        C's scheduler is NOT involved.
+        """
+        bootstrap_url = pm.remote_bootstrap_addr
+        num_remote_tokens = pm.remote_num_tokens or len(pm.prompt_token_ids)
+
+        try:
+            local_block_ids = pm.local_block_ids
+            d_session = f"{self.hostname}:{self.rpc_port}"
+
+            async with httpx.AsyncClient(timeout=60) as client:
+                resp = await client.post(
+                    f"{bootstrap_url}/push_blocks",
+                    json={
+                        "token_ids": pm.prompt_token_ids,
+                        "num_tokens": num_remote_tokens,
+                        "dst_block_ids": local_block_ids,
+                        "dst_base_addrs": self.kv_caches_base_addr,
+                        "dst_block_len": self.block_len,
+                        "dst_session": d_session,
+                    },
+                )
+                resp.raise_for_status()
+                result = resp.json()
+
+            matched = result.get("matched", 0)
+            pushed = result.get("pushed", False)
+
+            if matched > 0 and pushed:
+                logger.info("direct_push %s: %d blocks pushed from C", req_id, matched)
+            else:
+                logger.debug("direct_push %s: %d matched, pushed=%s", req_id, matched, pushed)
+                self.failed_recving_block_ids.update(local_block_ids)
+
+            self.finished_recving_reqs.add(req_id)
+
+        except Exception as e:
+            logger.error("direct_push %s failed: %s", req_id, e)
+            self.failed_recving_block_ids.update(pm.local_block_ids)
+            self.finished_recving_reqs.add(req_id)
+
     async def _start_load_kv(
         self, reqs_to_recv: dict[EngineId, dict[ReqId, PullReqMeta]]
     ):
@@ -1227,11 +1394,34 @@ class MooncakeConnectorWorker:
                 assert not send_meta.ready.is_set()
 
     def start_load_kv(self, metadata: MooncakeConnectorMetadata):
-        if not self.is_kv_producer and metadata.reqs_to_recv:
-            asyncio.run_coroutine_threadsafe(
-                self._start_load_kv(metadata.reqs_to_recv), self.receiver_loop
+        # Sync hash table to bootstrap server (for direct RDMA read queries)
+        if self.bootstrap_server is not None and (
+            metadata.hash_table_updates or metadata.hash_table_removals
+        ):
+            self.bootstrap_server.update_hash_table(
+                metadata.hash_table_updates, metadata.hash_table_removals
             )
 
+        if not self.is_kv_producer and metadata.reqs_to_recv:
+            # Split direct_read vs normal pull requests
+            direct_reqs: dict[EngineId, dict[ReqId, PullReqMeta]] = defaultdict(dict)
+            normal_reqs: dict[EngineId, dict[ReqId, PullReqMeta]] = defaultdict(dict)
+            for engine_id, pull_metas in metadata.reqs_to_recv.items():
+                for req_id, pm in pull_metas.items():
+                    if pm.direct_read:
+                        direct_reqs[engine_id][req_id] = pm
+                    else:
+                        normal_reqs[engine_id][req_id] = pm
+
+            if normal_reqs:
+                asyncio.run_coroutine_threadsafe(
+                    self._start_load_kv(normal_reqs), self.receiver_loop
+                )
+            if direct_reqs:
+                asyncio.run_coroutine_threadsafe(
+                    self._start_direct_read(direct_reqs), self.receiver_loop
+                )
+
         if not self.is_kv_consumer and (
             metadata.reqs_to_send or metadata.reqs_not_processed
         ):

third_party/vllm/vllm/distributed/kv_transfer/kv_connector/v1/mooncake/mooncake_utils.py

@@ -32,10 +32,53 @@ class EngineEntry:
     worker_addr: dict[int, dict[int, WorkerAddr]]
 
 
+class QueryBlocksRequest(BaseModel):
+    block_hashes: list[str] | None = None  # hex-encoded BlockHash values (legacy)
+    token_ids: list[int] | None = None     # raw token IDs for C-side hash lookup
+    num_tokens: int | None = None          # number of tokens to match prefix for
+    pin_token: str
+
+
+class QueryBlocksResponse(BaseModel):
+    block_ids: list[int | None]  # None = cache miss (prefix match stops)
+    kv_caches_base_addr: list[int]
+    block_len: int
+    hostname: str
+    rpc_port: int
+
+
+class PushBlocksRequest(BaseModel):
+    token_ids: list[int]
+    num_tokens: int
+    dst_block_ids: list[int]  # D's allocated block IDs for receiving
+    dst_base_addrs: list[int]  # D's kv_caches_base_addr
+    dst_block_len: int  # D's block_len
+    dst_session: str  # D's "hostname:rpc_port" for RDMA write
+
+
+class PushBlocksResponse(BaseModel):
+    matched: int
+    pushed: bool
+
+
+class EstimateHitRequest(BaseModel):
+    token_ids: list[int]
+    block_size: int = 512
+
+
+class EstimateHitResponse(BaseModel):
+    hit_tokens: int
+
+
+class UnpinBlocksRequest(BaseModel):
+    pin_token: str
+
+
 class MooncakeBootstrapServer:
     """
     A centralized server running on the global rank 0 prefiller worker.
     Prefiller workers register their connection info (IP, port, ranks) here.
+    Also serves block mapping queries for direct RDMA read.
     """
 
     def __init__(self, vllm_config: VllmConfig, host: str, port: int):
@@ -48,13 +91,23 @@ class MooncakeBootstrapServer:
         self.server_thread: threading.Thread | None = None
         self.server: uvicorn.Server | None = None
 
+        # Direct RDMA read support
+        self._hash_table: dict[str, int] = {}  # hex BlockHash → block_id
+        self._token_hash_table: dict[str, int] = {}  # str(hash(token_tuple)) → block_id
+        self._kv_info: dict | None = None
+        self._pinned: dict[str, list[int]] = {}
+        self._block_pool = None
+
     def __del__(self):
         self.shutdown()
 
     def _register_routes(self):
-        # All methods are async. No need to use lock to protect data.
         self.app.post("/register")(self.register_worker)
         self.app.get("/query", response_model=dict[int, EngineEntry])(self.query)
+        self.app.post("/query_blocks")(self.query_blocks)
+        self.app.post("/unpin_blocks")(self.unpin_blocks)
+        self.app.post("/push_blocks")(self.push_blocks)
+        self.app.post("/estimate_hit")(self.estimate_hit)
 
     def start(self):
         if self.server_thread:
@@ -125,3 +178,180 @@ class MooncakeBootstrapServer:
 
     async def query(self) -> dict[int, EngineEntry]:
         return self.workers
+
+    def set_worker_kv_info(
+        self,
+        kv_caches_base_addr: list[int],
+        block_len: int,
+        block_size: int,
+        hostname: str,
+        rpc_port: int,
+        transfer_engine=None,
+    ):
+        self._kv_info = {
+            "kv_caches_base_addr": kv_caches_base_addr,
+            "block_len": block_len,
+            "block_size": block_size,
+            "hostname": hostname,
+            "rpc_port": rpc_port,
+        }
+        self._transfer_engine = transfer_engine
+
+    def update_hash_table(
+        self,
+        updates: dict[str, int],
+        removals: set[str],
+    ):
+        for k in removals:
+            self._hash_table.pop(k, None)
+        self._hash_table.update(updates)
+
+    def set_block_pool(self, block_pool):
+        """Store reference to scheduler's block pool for token-based lookup."""
+        self._block_pool = block_pool
+
+    async def query_blocks(self, req: QueryBlocksRequest):
+        if self._kv_info is None:
+            raise HTTPException(503, "Worker KV info not registered yet")
+
+        block_ids: list[int | None] = []
+        pinned_ids: list[int] = []
+
+        if req.token_ids is not None and self._hash_table:
+            # Token-based lookup: compute hashes from tokens, match against synced hash table
+            block_ids, pinned_ids = self._lookup_by_tokens(
+                req.token_ids, req.num_tokens)
+        elif req.block_hashes is not None:
+            # Hash-based lookup (legacy)
+            for h in req.block_hashes:
+                bid = self._hash_table.get(h)
+                if bid is not None:
+                    block_ids.append(bid)
+                    pinned_ids.append(bid)
+                else:
+                    block_ids.append(None)
+                    break
+
+        logger.info(
+            "query_blocks: %d/%d matched (token_mode=%s, hash_table=%d)",
+            len(pinned_ids),
+            req.num_tokens // self._kv_info.get("block_size", 512)
+            if req.num_tokens else len(req.block_hashes or []),
+            req.token_ids is not None,
+            len(self._hash_table),
+        )
+
+        self._pinned[req.pin_token] = pinned_ids
+        return QueryBlocksResponse(
+            block_ids=block_ids,
+            kv_caches_base_addr=self._kv_info["kv_caches_base_addr"],
+            block_len=self._kv_info["block_len"],
+            hostname=self._kv_info["hostname"],
+            rpc_port=self._kv_info["rpc_port"],
+        )
+
+    def _lookup_by_tokens(
+        self, token_ids: list[int], num_tokens: int | None
+    ) -> tuple[list[int | None], list[int]]:
+        """Look up cached blocks by computing hashes using the synced hash table.
+
+        Uses module-level NONE_HASH (accessed via module ref to get latest value
+        after init_none_hash is called).
+        """
+        import vllm.v1.core.kv_cache_utils as kv_utils
+        from vllm.utils.hashing import sha256
+
+        block_size = self._kv_info.get("block_size", 512) if self._kv_info else 512
+        n = num_tokens or len(token_ids)
+        n = min(n, len(token_ids))
+        num_blocks = n // block_size
+
+        block_ids: list[int | None] = []
+        pinned_ids: list[int] = []
+        prev_hash = kv_utils.NONE_HASH  # module-level ref, always current
+
+        for i in range(num_blocks):
+            block_tokens = tuple(token_ids[i * block_size:(i + 1) * block_size])
+            block_hash = kv_utils.hash_block_tokens(
+                sha256, prev_hash, block_tokens, None)
+            prev_hash = block_hash
+
+            bid = self._hash_table.get(block_hash.hex())
+            if i == 0:
+                table_sample = next(iter(self._hash_table)) if self._hash_table else "empty"
+                logger.info(
+                    "_lookup: hash=%s NONE=%s tbl=%s",
+                    block_hash.hex()[:12], kv_utils.NONE_HASH.hex()[:12], table_sample[:12])
+            if bid is not None:
+                block_ids.append(bid)
+                pinned_ids.append(bid)
+            else:
+                if i == 0:
+                    block_ids.append(None)
+                break
+
+        return block_ids, pinned_ids
+
+    async def unpin_blocks(self, req: UnpinBlocksRequest):
+        self._pinned.pop(req.pin_token, None)
+        return {"status": "ok"}
+
+    async def estimate_hit(self, req: EstimateHitRequest):
+        """Read-only probe: how many prefix-contiguous tokens are cached?
+
+        Reuses _lookup_by_tokens (proven to work with push_blocks) instead
+        of reimplementing hash computation.
+        """
+        if self._kv_info is None:
+            raise HTTPException(503, "Worker KV info not registered yet")
+
+        if not self._hash_table:
+            return EstimateHitResponse(hit_tokens=0)
+
+        block_ids, _ = self._lookup_by_tokens(req.token_ids, None)
+        hit_blocks = sum(1 for b in block_ids if b is not None)
+        block_size = self._kv_info.get("block_size", 512)
+        hit_tokens = hit_blocks * block_size
+
+        logger.info("estimate_hit: %d/%d blocks hit (%d tokens, tbl=%d)",
+                    hit_blocks, len(req.token_ids) // block_size,
+                    hit_tokens, len(self._hash_table))
+        return EstimateHitResponse(hit_tokens=hit_tokens)
+
+    async def push_blocks(self, req: PushBlocksRequest):
+        """Query matching blocks by token_ids, then PUSH them to D via RDMA write."""
+        if self._kv_info is None or self._transfer_engine is None:
+            raise HTTPException(503, "Worker not ready")
+
+        block_ids, _ = self._lookup_by_tokens(req.token_ids, req.num_tokens)
+        matched_src = [b for b in block_ids if b is not None]
+        num_matched = len(matched_src)
+
+        if num_matched == 0:
+            logger.info("push_blocks: 0 matched")
+            return PushBlocksResponse(matched=0, pushed=False)
+
+        matched_dst = req.dst_block_ids[:num_matched]
+        src_base = self._kv_info["kv_caches_base_addr"]
+        src_block_len = self._kv_info["block_len"]
+
+        src_ptrs: list[int] = []
+        dst_ptrs: list[int] = []
+        lengths: list[int] = []
+
+        for src_layer, dst_layer in zip(src_base, req.dst_base_addrs):
+            for s_bid, d_bid in zip(matched_src, matched_dst):
+                src_ptrs.append(src_layer + s_bid * src_block_len)
+                dst_ptrs.append(dst_layer + d_bid * req.dst_block_len)
+                lengths.append(src_block_len)
+
+        import asyncio
+        loop = asyncio.get_event_loop()
+        ret = await loop.run_in_executor(
+            None,
+            self._transfer_engine.batch_transfer_sync_write,
+            req.dst_session, src_ptrs, dst_ptrs, lengths,
+        )
+
+        logger.info("push_blocks: %d matched, push ret=%d", num_matched, ret)
+        return PushBlocksResponse(matched=num_matched, pushed=(ret == 0))

third_party/vllm/vllm/v1/core/sched/scheduler.py

@@ -234,6 +234,8 @@ class Scheduler(SchedulerInterface):
             hash_block_size=self.block_size,
             metrics_collector=self.kv_metrics_collector,
         )
+        if self.connector is not None and hasattr(self.connector, "set_block_pool"):
+            self.connector.set_block_pool(self.kv_cache_manager.block_pool)
         self.use_pp = self.parallel_config.pipeline_parallel_size > 1
         self.use_v2_model_runner = envs.VLLM_USE_V2_MODEL_RUNNER
 
@@ -2103,15 +2105,23 @@ class Scheduler(SchedulerInterface):
             req = self.requests[req_id]
             if req.status == RequestStatus.WAITING_FOR_REMOTE_KVS:
                 self.finished_recving_kv_req_ids.add(req_id)
-            else:
-                assert RequestStatus.is_finished(req.status)
+            elif RequestStatus.is_finished(req.status):
                 self._free_blocks(self.requests[req_id])
+            else:
+                logger.debug(
+                    "finished_recving for %s in status %s (partial remote prefill?)",
+                    req_id, req.status)
         for req_id in kv_connector_output.finished_sending or ():
             logger.debug("Finished sending KV transfer for request %s", req_id)
             if req_id not in self.requests:
                 logger.warning("Skipping finished_sending for unknown request %s (already aborted?)", req_id)
                 continue
+            sent_block_ids: set[int] = set()
+            for group in self.kv_cache_manager.get_block_ids(req_id):
+                sent_block_ids.update(group)
             self._free_blocks(self.requests[req_id])
+            if sent_block_ids:
+                self.kv_cache_manager.evict_blocks(sent_block_ids)
 
     def _update_requests_with_invalid_blocks(
         self,

uv.lock

@@ -0,0 +1,384 @@
+version = 1
+revision = 3
+requires-python = ">=3.10"
+resolution-markers = [
+    "python_full_version >= '3.11'",
+    "python_full_version < '3.11'",
+]
+
+[[package]]
+name = "agentic-kv"
+version = "0.1.0"
+source = { editable = "." }
+dependencies = [
+    { name = "httpx" },
+    { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
+    { name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
+]
+
+[package.optional-dependencies]
+dev = [
+    { name = "pytest" },
+]
+
+[package.metadata]
+requires-dist = [
+    { name = "httpx", specifier = ">=0.27" },
+    { name = "numpy", specifier = ">=1.24" },
+    { name = "pytest", marker = "extra == 'dev'" },
+]
+provides-extras = ["dev"]
+
+[[package]]
+name = "anyio"
+version = "4.13.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "exceptiongroup", marker = "python_full_version < '3.11'" },
+    { name = "idna" },
+    { name = "typing-extensions", marker = "python_full_version < '3.13'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/19/14/2c5dd9f512b66549ae92767a9c7b330ae88e1932ca57876909410251fe13/anyio-4.13.0.tar.gz", hash = "sha256:334b70e641fd2221c1505b3890c69882fe4a2df910cba14d97019b90b24439dc", size = 231622, upload-time = "2026-03-24T12:59:09.671Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/da/42/e921fccf5015463e32a3cf6ee7f980a6ed0f395ceeaa45060b61d86486c2/anyio-4.13.0-py3-none-any.whl", hash = "sha256:08b310f9e24a9594186fd75b4f73f4a4152069e3853f1ed8bfbf58369f4ad708", size = 114353, upload-time = "2026-03-24T12:59:08.246Z" },
+]
+
+[[package]]
+name = "certifi"
+version = "2026.5.20"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/f3/ce/ee2ecad540810a79593028e88299baeae54d346cc7a0d94b6199988b89b1/certifi-2026.5.20.tar.gz", hash = "sha256:69dea482ab64caa7b9f6aba1c6bf48bb6a5448d1c0f1b17ab42ad8c763a5344d", size = 135422, upload-time = "2026-05-20T11:46:50.073Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/59/8c/57e832b7af6d7c5abe66eb3fbe3a3a32f4d11ea23a1aa7131371035be991/certifi-2026.5.20-py3-none-any.whl", hash = "sha256:3c52e209ba0a4ad7aebe60436a4ab349c39e1e602e8c134221e546902ad25897", size = 134134, upload-time = "2026-05-20T11:46:48.578Z" },
+]
+
+[[package]]
+name = "colorama"
+version = "0.4.6"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
+]
+
+[[package]]
+name = "exceptiongroup"
+version = "1.3.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "typing-extensions", marker = "python_full_version < '3.11'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/50/79/66800aadf48771f6b62f7eb014e352e5d06856655206165d775e675a02c9/exceptiongroup-1.3.1.tar.gz", hash = "sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219", size = 30371, upload-time = "2025-11-21T23:01:54.787Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/8a/0e/97c33bf5009bdbac74fd2beace167cab3f978feb69cc36f1ef79360d6c4e/exceptiongroup-1.3.1-py3-none-any.whl", hash = "sha256:a7a39a3bd276781e98394987d3a5701d0c4edffb633bb7a5144577f82c773598", size = 16740, upload-time = "2025-11-21T23:01:53.443Z" },
+]
+
+[[package]]
+name = "h11"
+version = "0.16.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/01/ee/02a2c011bdab74c6fb3c75474d40b3052059d95df7e73351460c8588d963/h11-0.16.0.tar.gz", hash = "sha256:4e35b956cf45792e4caa5885e69fba00bdbc6ffafbfa020300e549b208ee5ff1", size = 101250, upload-time = "2025-04-24T03:35:25.427Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", hash = "sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", size = 37515, upload-time = "2025-04-24T03:35:24.344Z" },
+]
+
+[[package]]
+name = "httpcore"
+version = "1.0.9"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "certifi" },
+    { name = "h11" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/06/94/82699a10bca87a5556c9c59b5963f2d039dbd239f25bc2a63907a05a14cb/httpcore-1.0.9.tar.gz", hash = "sha256:6e34463af53fd2ab5d807f399a9b45ea31c3dfa2276f15a2c3f00afff6e176e8", size = 85484, upload-time = "2025-04-24T22:06:22.219Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/7e/f5/f66802a942d491edb555dd61e3a9961140fd64c90bce1eafd741609d334d/httpcore-1.0.9-py3-none-any.whl", hash = "sha256:2d400746a40668fc9dec9810239072b40b4484b640a8c38fd654a024c7a1bf55", size = 78784, upload-time = "2025-04-24T22:06:20.566Z" },
+]
+
+[[package]]
+name = "httpx"
+version = "0.28.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "anyio" },
+    { name = "certifi" },
+    { name = "httpcore" },
+    { name = "idna" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/b1/df/48c586a5fe32a0f01324ee087459e112ebb7224f646c0b5023f5e79e9956/httpx-0.28.1.tar.gz", hash = "sha256:75e98c5f16b0f35b567856f597f06ff2270a374470a5c2392242528e3e3e42fc", size = 141406, upload-time = "2024-12-06T15:37:23.222Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", hash = "sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", size = 73517, upload-time = "2024-12-06T15:37:21.509Z" },
+]
+
+[[package]]
+name = "idna"
+version = "3.16"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/1a/88/bcf9709822fe69d02c2a6a77956c98ce6ea8ca8767a9aadcedc7eb6a2390/idna-3.16.tar.gz", hash = "sha256:d7a6da03db833450fca25d2358ac9ff06cd624577a4aea3a596d5c0f77b8e03d", size = 203770, upload-time = "2026-05-22T00:16:18.781Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/94/16/70255075a9859a0e3adb789b68ceb0e210dec03934245fd98d248226572f/idna-3.16-py3-none-any.whl", hash = "sha256:cc246e3a3f89580c3a951b5ad298ca4638078b2cdd4f115654332b5c26daded5", size = 74165, upload-time = "2026-05-22T00:16:16.698Z" },
+]
+
+[[package]]
+name = "iniconfig"
+version = "2.3.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
+]
+
+[[package]]
+name = "numpy"
+version = "2.2.6"
+source = { registry = "https://pypi.org/simple" }
+resolution-markers = [
+    "python_full_version < '3.11'",
+]
+sdist = { url = "https://files.pythonhosted.org/packages/76/21/7d2a95e4bba9dc13d043ee156a356c0a8f0c6309dff6b21b4d71a073b8a8/numpy-2.2.6.tar.gz", hash = "sha256:e29554e2bef54a90aa5cc07da6ce955accb83f21ab5de01a62c8478897b264fd", size = 20276440, upload-time = "2025-05-17T22:38:04.611Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/9a/3e/ed6db5be21ce87955c0cbd3009f2803f59fa08df21b5df06862e2d8e2bdd/numpy-2.2.6-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:b412caa66f72040e6d268491a59f2c43bf03eb6c96dd8f0307829feb7fa2b6fb", size = 21165245, upload-time = "2025-05-17T21:27:58.555Z" },
+    { url = "https://files.pythonhosted.org/packages/22/c2/4b9221495b2a132cc9d2eb862e21d42a009f5a60e45fc44b00118c174bff/numpy-2.2.6-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:8e41fd67c52b86603a91c1a505ebaef50b3314de0213461c7a6e99c9a3beff90", size = 14360048, upload-time = "2025-05-17T21:28:21.406Z" },
+    { url = "https://files.pythonhosted.org/packages/fd/77/dc2fcfc66943c6410e2bf598062f5959372735ffda175b39906d54f02349/numpy-2.2.6-cp310-cp310-macosx_14_0_arm64.whl", hash = "sha256:37e990a01ae6ec7fe7fa1c26c55ecb672dd98b19c3d0e1d1f326fa13cb38d163", size = 5340542, upload-time = "2025-05-17T21:28:30.931Z" },
+    { url = "https://files.pythonhosted.org/packages/7a/4f/1cb5fdc353a5f5cc7feb692db9b8ec2c3d6405453f982435efc52561df58/numpy-2.2.6-cp310-cp310-macosx_14_0_x86_64.whl", hash = "sha256:5a6429d4be8ca66d889b7cf70f536a397dc45ba6faeb5f8c5427935d9592e9cf", size = 6878301, upload-time = "2025-05-17T21:28:41.613Z" },
+    { url = "https://files.pythonhosted.org/packages/eb/17/96a3acd228cec142fcb8723bd3cc39c2a474f7dcf0a5d16731980bcafa95/numpy-2.2.6-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:efd28d4e9cd7d7a8d39074a4d44c63eda73401580c5c76acda2ce969e0a38e83", size = 14297320, upload-time = "2025-05-17T21:29:02.78Z" },
+    { url = "https://files.pythonhosted.org/packages/b4/63/3de6a34ad7ad6646ac7d2f55ebc6ad439dbbf9c4370017c50cf403fb19b5/numpy-2.2.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:fc7b73d02efb0e18c000e9ad8b83480dfcd5dfd11065997ed4c6747470ae8915", size = 16801050, upload-time = "2025-05-17T21:29:27.675Z" },
+    { url = "https://files.pythonhosted.org/packages/07/b6/89d837eddef52b3d0cec5c6ba0456c1bf1b9ef6a6672fc2b7873c3ec4e2e/numpy-2.2.6-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:74d4531beb257d2c3f4b261bfb0fc09e0f9ebb8842d82a7b4209415896adc680", size = 15807034, upload-time = "2025-05-17T21:29:51.102Z" },
+    { url = "https://files.pythonhosted.org/packages/01/c8/dc6ae86e3c61cfec1f178e5c9f7858584049b6093f843bca541f94120920/numpy-2.2.6-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:8fc377d995680230e83241d8a96def29f204b5782f371c532579b4f20607a289", size = 18614185, upload-time = "2025-05-17T21:30:18.703Z" },
+    { url = "https://files.pythonhosted.org/packages/5b/c5/0064b1b7e7c89137b471ccec1fd2282fceaae0ab3a9550f2568782d80357/numpy-2.2.6-cp310-cp310-win32.whl", hash = "sha256:b093dd74e50a8cba3e873868d9e93a85b78e0daf2e98c6797566ad8044e8363d", size = 6527149, upload-time = "2025-05-17T21:30:29.788Z" },
+    { url = "https://files.pythonhosted.org/packages/a3/dd/4b822569d6b96c39d1215dbae0582fd99954dcbcf0c1a13c61783feaca3f/numpy-2.2.6-cp310-cp310-win_amd64.whl", hash = "sha256:f0fd6321b839904e15c46e0d257fdd101dd7f530fe03fd6359c1ea63738703f3", size = 12904620, upload-time = "2025-05-17T21:30:48.994Z" },
+    { url = "https://files.pythonhosted.org/packages/da/a8/4f83e2aa666a9fbf56d6118faaaf5f1974d456b1823fda0a176eff722839/numpy-2.2.6-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:f9f1adb22318e121c5c69a09142811a201ef17ab257a1e66ca3025065b7f53ae", size = 21176963, upload-time = "2025-05-17T21:31:19.36Z" },
+    { url = "https://files.pythonhosted.org/packages/b3/2b/64e1affc7972decb74c9e29e5649fac940514910960ba25cd9af4488b66c/numpy-2.2.6-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:c820a93b0255bc360f53eca31a0e676fd1101f673dda8da93454a12e23fc5f7a", size = 14406743, upload-time = "2025-05-17T21:31:41.087Z" },
+    { url = "https://files.pythonhosted.org/packages/4a/9f/0121e375000b5e50ffdd8b25bf78d8e1a5aa4cca3f185d41265198c7b834/numpy-2.2.6-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:3d70692235e759f260c3d837193090014aebdf026dfd167834bcba43e30c2a42", size = 5352616, upload-time = "2025-05-17T21:31:50.072Z" },
+    { url = "https://files.pythonhosted.org/packages/31/0d/b48c405c91693635fbe2dcd7bc84a33a602add5f63286e024d3b6741411c/numpy-2.2.6-cp311-cp311-macosx_14_0_x86_64.whl", hash = "sha256:481b49095335f8eed42e39e8041327c05b0f6f4780488f61286ed3c01368d491", size = 6889579, upload-time = "2025-05-17T21:32:01.712Z" },
+    { url = "https://files.pythonhosted.org/packages/52/b8/7f0554d49b565d0171eab6e99001846882000883998e7b7d9f0d98b1f934/numpy-2.2.6-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b64d8d4d17135e00c8e346e0a738deb17e754230d7e0810ac5012750bbd85a5a", size = 14312005, upload-time = "2025-05-17T21:32:23.332Z" },
+    { url = "https://files.pythonhosted.org/packages/b3/dd/2238b898e51bd6d389b7389ffb20d7f4c10066d80351187ec8e303a5a475/numpy-2.2.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ba10f8411898fc418a521833e014a77d3ca01c15b0c6cdcce6a0d2897e6dbbdf", size = 16821570, upload-time = "2025-05-17T21:32:47.991Z" },
+    { url = "https://files.pythonhosted.org/packages/83/6c/44d0325722cf644f191042bf47eedad61c1e6df2432ed65cbe28509d404e/numpy-2.2.6-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:bd48227a919f1bafbdda0583705e547892342c26fb127219d60a5c36882609d1", size = 15818548, upload-time = "2025-05-17T21:33:11.728Z" },
+    { url = "https://files.pythonhosted.org/packages/ae/9d/81e8216030ce66be25279098789b665d49ff19eef08bfa8cb96d4957f422/numpy-2.2.6-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:9551a499bf125c1d4f9e250377c1ee2eddd02e01eac6644c080162c0c51778ab", size = 18620521, upload-time = "2025-05-17T21:33:39.139Z" },
+    { url = "https://files.pythonhosted.org/packages/6a/fd/e19617b9530b031db51b0926eed5345ce8ddc669bb3bc0044b23e275ebe8/numpy-2.2.6-cp311-cp311-win32.whl", hash = "sha256:0678000bb9ac1475cd454c6b8c799206af8107e310843532b04d49649c717a47", size = 6525866, upload-time = "2025-05-17T21:33:50.273Z" },
+    { url = "https://files.pythonhosted.org/packages/31/0a/f354fb7176b81747d870f7991dc763e157a934c717b67b58456bc63da3df/numpy-2.2.6-cp311-cp311-win_amd64.whl", hash = "sha256:e8213002e427c69c45a52bbd94163084025f533a55a59d6f9c5b820774ef3303", size = 12907455, upload-time = "2025-05-17T21:34:09.135Z" },
+    { url = "https://files.pythonhosted.org/packages/82/5d/c00588b6cf18e1da539b45d3598d3557084990dcc4331960c15ee776ee41/numpy-2.2.6-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:41c5a21f4a04fa86436124d388f6ed60a9343a6f767fced1a8a71c3fbca038ff", size = 20875348, upload-time = "2025-05-17T21:34:39.648Z" },
+    { url = "https://files.pythonhosted.org/packages/66/ee/560deadcdde6c2f90200450d5938f63a34b37e27ebff162810f716f6a230/numpy-2.2.6-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:de749064336d37e340f640b05f24e9e3dd678c57318c7289d222a8a2f543e90c", size = 14119362, upload-time = "2025-05-17T21:35:01.241Z" },
+    { url = "https://files.pythonhosted.org/packages/3c/65/4baa99f1c53b30adf0acd9a5519078871ddde8d2339dc5a7fde80d9d87da/numpy-2.2.6-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:894b3a42502226a1cac872f840030665f33326fc3dac8e57c607905773cdcde3", size = 5084103, upload-time = "2025-05-17T21:35:10.622Z" },
+    { url = "https://files.pythonhosted.org/packages/cc/89/e5a34c071a0570cc40c9a54eb472d113eea6d002e9ae12bb3a8407fb912e/numpy-2.2.6-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:71594f7c51a18e728451bb50cc60a3ce4e6538822731b2933209a1f3614e9282", size = 6625382, upload-time = "2025-05-17T21:35:21.414Z" },
+    { url = "https://files.pythonhosted.org/packages/f8/35/8c80729f1ff76b3921d5c9487c7ac3de9b2a103b1cd05e905b3090513510/numpy-2.2.6-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f2618db89be1b4e05f7a1a847a9c1c0abd63e63a1607d892dd54668dd92faf87", size = 14018462, upload-time = "2025-05-17T21:35:42.174Z" },
+    { url = "https://files.pythonhosted.org/packages/8c/3d/1e1db36cfd41f895d266b103df00ca5b3cbe965184df824dec5c08c6b803/numpy-2.2.6-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:fd83c01228a688733f1ded5201c678f0c53ecc1006ffbc404db9f7a899ac6249", size = 16527618, upload-time = "2025-05-17T21:36:06.711Z" },
+    { url = "https://files.pythonhosted.org/packages/61/c6/03ed30992602c85aa3cd95b9070a514f8b3c33e31124694438d88809ae36/numpy-2.2.6-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:37c0ca431f82cd5fa716eca9506aefcabc247fb27ba69c5062a6d3ade8cf8f49", size = 15505511, upload-time = "2025-05-17T21:36:29.965Z" },
+    { url = "https://files.pythonhosted.org/packages/b7/25/5761d832a81df431e260719ec45de696414266613c9ee268394dd5ad8236/numpy-2.2.6-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:fe27749d33bb772c80dcd84ae7e8df2adc920ae8297400dabec45f0dedb3f6de", size = 18313783, upload-time = "2025-05-17T21:36:56.883Z" },
+    { url = "https://files.pythonhosted.org/packages/57/0a/72d5a3527c5ebffcd47bde9162c39fae1f90138c961e5296491ce778e682/numpy-2.2.6-cp312-cp312-win32.whl", hash = "sha256:4eeaae00d789f66c7a25ac5f34b71a7035bb474e679f410e5e1a94deb24cf2d4", size = 6246506, upload-time = "2025-05-17T21:37:07.368Z" },
+    { url = "https://files.pythonhosted.org/packages/36/fa/8c9210162ca1b88529ab76b41ba02d433fd54fecaf6feb70ef9f124683f1/numpy-2.2.6-cp312-cp312-win_amd64.whl", hash = "sha256:c1f9540be57940698ed329904db803cf7a402f3fc200bfe599334c9bd84a40b2", size = 12614190, upload-time = "2025-05-17T21:37:26.213Z" },
+    { url = "https://files.pythonhosted.org/packages/f9/5c/6657823f4f594f72b5471f1db1ab12e26e890bb2e41897522d134d2a3e81/numpy-2.2.6-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:0811bb762109d9708cca4d0b13c4f67146e3c3b7cf8d34018c722adb2d957c84", size = 20867828, upload-time = "2025-05-17T21:37:56.699Z" },
+    { url = "https://files.pythonhosted.org/packages/dc/9e/14520dc3dadf3c803473bd07e9b2bd1b69bc583cb2497b47000fed2fa92f/numpy-2.2.6-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:287cc3162b6f01463ccd86be154f284d0893d2b3ed7292439ea97eafa8170e0b", size = 14143006, upload-time = "2025-05-17T21:38:18.291Z" },
+    { url = "https://files.pythonhosted.org/packages/4f/06/7e96c57d90bebdce9918412087fc22ca9851cceaf5567a45c1f404480e9e/numpy-2.2.6-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:f1372f041402e37e5e633e586f62aa53de2eac8d98cbfb822806ce4bbefcb74d", size = 5076765, upload-time = "2025-05-17T21:38:27.319Z" },
+    { url = "https://files.pythonhosted.org/packages/73/ed/63d920c23b4289fdac96ddbdd6132e9427790977d5457cd132f18e76eae0/numpy-2.2.6-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:55a4d33fa519660d69614a9fad433be87e5252f4b03850642f88993f7b2ca566", size = 6617736, upload-time = "2025-05-17T21:38:38.141Z" },
+    { url = "https://files.pythonhosted.org/packages/85/c5/e19c8f99d83fd377ec8c7e0cf627a8049746da54afc24ef0a0cb73d5dfb5/numpy-2.2.6-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f92729c95468a2f4f15e9bb94c432a9229d0d50de67304399627a943201baa2f", size = 14010719, upload-time = "2025-05-17T21:38:58.433Z" },
+    { url = "https://files.pythonhosted.org/packages/19/49/4df9123aafa7b539317bf6d342cb6d227e49f7a35b99c287a6109b13dd93/numpy-2.2.6-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1bc23a79bfabc5d056d106f9befb8d50c31ced2fbc70eedb8155aec74a45798f", size = 16526072, upload-time = "2025-05-17T21:39:22.638Z" },
+    { url = "https://files.pythonhosted.org/packages/b2/6c/04b5f47f4f32f7c2b0e7260442a8cbcf8168b0e1a41ff1495da42f42a14f/numpy-2.2.6-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:e3143e4451880bed956e706a3220b4e5cf6172ef05fcc397f6f36a550b1dd868", size = 15503213, upload-time = "2025-05-17T21:39:45.865Z" },
+    { url = "https://files.pythonhosted.org/packages/17/0a/5cd92e352c1307640d5b6fec1b2ffb06cd0dabe7d7b8227f97933d378422/numpy-2.2.6-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:b4f13750ce79751586ae2eb824ba7e1e8dba64784086c98cdbbcc6a42112ce0d", size = 18316632, upload-time = "2025-05-17T21:40:13.331Z" },
+    { url = "https://files.pythonhosted.org/packages/f0/3b/5cba2b1d88760ef86596ad0f3d484b1cbff7c115ae2429678465057c5155/numpy-2.2.6-cp313-cp313-win32.whl", hash = "sha256:5beb72339d9d4fa36522fc63802f469b13cdbe4fdab4a288f0c441b74272ebfd", size = 6244532, upload-time = "2025-05-17T21:43:46.099Z" },
+    { url = "https://files.pythonhosted.org/packages/cb/3b/d58c12eafcb298d4e6d0d40216866ab15f59e55d148a5658bb3132311fcf/numpy-2.2.6-cp313-cp313-win_amd64.whl", hash = "sha256:b0544343a702fa80c95ad5d3d608ea3599dd54d4632df855e4c8d24eb6ecfa1c", size = 12610885, upload-time = "2025-05-17T21:44:05.145Z" },
+    { url = "https://files.pythonhosted.org/packages/6b/9e/4bf918b818e516322db999ac25d00c75788ddfd2d2ade4fa66f1f38097e1/numpy-2.2.6-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:0bca768cd85ae743b2affdc762d617eddf3bcf8724435498a1e80132d04879e6", size = 20963467, upload-time = "2025-05-17T21:40:44Z" },
+    { url = "https://files.pythonhosted.org/packages/61/66/d2de6b291507517ff2e438e13ff7b1e2cdbdb7cb40b3ed475377aece69f9/numpy-2.2.6-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:fc0c5673685c508a142ca65209b4e79ed6740a4ed6b2267dbba90f34b0b3cfda", size = 14225144, upload-time = "2025-05-17T21:41:05.695Z" },
+    { url = "https://files.pythonhosted.org/packages/e4/25/480387655407ead912e28ba3a820bc69af9adf13bcbe40b299d454ec011f/numpy-2.2.6-cp313-cp313t-macosx_14_0_arm64.whl", hash = "sha256:5bd4fc3ac8926b3819797a7c0e2631eb889b4118a9898c84f585a54d475b7e40", size = 5200217, upload-time = "2025-05-17T21:41:15.903Z" },
+    { url = "https://files.pythonhosted.org/packages/aa/4a/6e313b5108f53dcbf3aca0c0f3e9c92f4c10ce57a0a721851f9785872895/numpy-2.2.6-cp313-cp313t-macosx_14_0_x86_64.whl", hash = "sha256:fee4236c876c4e8369388054d02d0e9bb84821feb1a64dd59e137e6511a551f8", size = 6712014, upload-time = "2025-05-17T21:41:27.321Z" },
+    { url = "https://files.pythonhosted.org/packages/b7/30/172c2d5c4be71fdf476e9de553443cf8e25feddbe185e0bd88b096915bcc/numpy-2.2.6-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e1dda9c7e08dc141e0247a5b8f49cf05984955246a327d4c48bda16821947b2f", size = 14077935, upload-time = "2025-05-17T21:41:49.738Z" },
+    { url = "https://files.pythonhosted.org/packages/12/fb/9e743f8d4e4d3c710902cf87af3512082ae3d43b945d5d16563f26ec251d/numpy-2.2.6-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f447e6acb680fd307f40d3da4852208af94afdfab89cf850986c3ca00562f4fa", size = 16600122, upload-time = "2025-05-17T21:42:14.046Z" },
+    { url = "https://files.pythonhosted.org/packages/12/75/ee20da0e58d3a66f204f38916757e01e33a9737d0b22373b3eb5a27358f9/numpy-2.2.6-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:389d771b1623ec92636b0786bc4ae56abafad4a4c513d36a55dce14bd9ce8571", size = 15586143, upload-time = "2025-05-17T21:42:37.464Z" },
+    { url = "https://files.pythonhosted.org/packages/76/95/bef5b37f29fc5e739947e9ce5179ad402875633308504a52d188302319c8/numpy-2.2.6-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:8e9ace4a37db23421249ed236fdcdd457d671e25146786dfc96835cd951aa7c1", size = 18385260, upload-time = "2025-05-17T21:43:05.189Z" },
+    { url = "https://files.pythonhosted.org/packages/09/04/f2f83279d287407cf36a7a8053a5abe7be3622a4363337338f2585e4afda/numpy-2.2.6-cp313-cp313t-win32.whl", hash = "sha256:038613e9fb8c72b0a41f025a7e4c3f0b7a1b5d768ece4796b674c8f3fe13efff", size = 6377225, upload-time = "2025-05-17T21:43:16.254Z" },
+    { url = "https://files.pythonhosted.org/packages/67/0e/35082d13c09c02c011cf21570543d202ad929d961c02a147493cb0c2bdf5/numpy-2.2.6-cp313-cp313t-win_amd64.whl", hash = "sha256:6031dd6dfecc0cf9f668681a37648373bddd6421fff6c66ec1624eed0180ee06", size = 12771374, upload-time = "2025-05-17T21:43:35.479Z" },
+    { url = "https://files.pythonhosted.org/packages/9e/3b/d94a75f4dbf1ef5d321523ecac21ef23a3cd2ac8b78ae2aac40873590229/numpy-2.2.6-pp310-pypy310_pp73-macosx_10_15_x86_64.whl", hash = "sha256:0b605b275d7bd0c640cad4e5d30fa701a8d59302e127e5f79138ad62762c3e3d", size = 21040391, upload-time = "2025-05-17T21:44:35.948Z" },
+    { url = "https://files.pythonhosted.org/packages/17/f4/09b2fa1b58f0fb4f7c7963a1649c64c4d315752240377ed74d9cd878f7b5/numpy-2.2.6-pp310-pypy310_pp73-macosx_14_0_x86_64.whl", hash = "sha256:7befc596a7dc9da8a337f79802ee8adb30a552a94f792b9c9d18c840055907db", size = 6786754, upload-time = "2025-05-17T21:44:47.446Z" },
+    { url = "https://files.pythonhosted.org/packages/af/30/feba75f143bdc868a1cc3f44ccfa6c4b9ec522b36458e738cd00f67b573f/numpy-2.2.6-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ce47521a4754c8f4593837384bd3424880629f718d87c5d44f8ed763edd63543", size = 16643476, upload-time = "2025-05-17T21:45:11.871Z" },
+    { url = "https://files.pythonhosted.org/packages/37/48/ac2a9584402fb6c0cd5b5d1a91dcf176b15760130dd386bbafdbfe3640bf/numpy-2.2.6-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:d042d24c90c41b54fd506da306759e06e568864df8ec17ccc17e9e884634fd00", size = 12812666, upload-time = "2025-05-17T21:45:31.426Z" },
+]
+
+[[package]]
+name = "numpy"
+version = "2.4.6"
+source = { registry = "https://pypi.org/simple" }
+resolution-markers = [
+    "python_full_version >= '3.11'",
+]
+sdist = { url = "https://files.pythonhosted.org/packages/d0/ad/fed0499ce6a338d2a03ebae59cd15093910c8875328855781952abf6c2fe/numpy-2.4.6.tar.gz", hash = "sha256:f3a3570c4a2a16746ac2c31a7c7c7b0c186b95ce902e33db6f28094ed7387dda", size = 20735807, upload-time = "2026-05-18T23:37:14.07Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/b3/49/ec46835a70be8fa6446c495126ac84fdb28cb2558e1620ffb87a10c8b64c/numpy-2.4.6-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:0280e0356c0829a18d9de1cb7eee50ec22ca639878d7240307ca0943d73cd2c4", size = 16969194, upload-time = "2026-05-18T23:33:13.503Z" },
+    { url = "https://files.pythonhosted.org/packages/0e/0d/f5957185c0ee2f3e12f78715aa9e3b353fd83633316c8532b38faa37e3f6/numpy-2.4.6-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:110f8b71aacb688ec69062bb7f6938a0f8acb01b7c1c4beb453c65b6d234584d", size = 14964111, upload-time = "2026-05-18T23:33:17.795Z" },
+    { url = "https://files.pythonhosted.org/packages/ad/40/40a40ee0ddf7ceb782c49af278894b686e586d65d8c1889c8b5da01a3d7d/numpy-2.4.6-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:4cfe66903cc32a9921a6733d96b19bb6abf310397581bbad89c228f5abaf0ee8", size = 5469159, upload-time = "2026-05-18T23:33:20.654Z" },
+    { url = "https://files.pythonhosted.org/packages/63/13/f9a8046535cb21deae82f8d03de9617e08882d274fad2539630761888228/numpy-2.4.6-cp311-cp311-macosx_14_0_x86_64.whl", hash = "sha256:8155154c7c691289fe18f510b5d4657c68c67989f293f0535a91360392ff6538", size = 6798936, upload-time = "2026-05-18T23:33:22.987Z" },
+    { url = "https://files.pythonhosted.org/packages/33/a8/6fa8c1a345a8c85dbb21932c447bee07c30a2c2a3f31e369c0a84b300147/numpy-2.4.6-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0ab0a9c4ffb1a6d95ef519fe4247dba8eb6b18ad93999f76b7f657039acabd47", size = 15966692, upload-time = "2026-05-18T23:33:26.62Z" },
+    { url = "https://files.pythonhosted.org/packages/02/03/74fe2a4cb3817d94d86402f2506554130a2f01414e299b5a843e5a8a957f/numpy-2.4.6-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:89cd468399cfd2504718f0ba50e410dca55a170b61a02ad92bb18c8a65186e93", size = 16918164, upload-time = "2026-05-18T23:33:29.955Z" },
+    { url = "https://files.pythonhosted.org/packages/c5/80/3615be3313f7e7696609bc194b9f0101da809df79e859bdb84e0cd043f46/numpy-2.4.6-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:c2d37ab77531417474168eb79d6d80b14f821a966818505d03013d0833edb7a8", size = 17322877, upload-time = "2026-05-18T23:33:34.724Z" },
+    { url = "https://files.pythonhosted.org/packages/ca/ac/a691e0fe2675e370d0e08ff905adc49a1c8830e8cae03efe4477e92cd55d/numpy-2.4.6-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f407cb6b8e9d6d8c626bc73c945db1706035af8fd632295547bf1c9e46d092d6", size = 18651487, upload-time = "2026-05-18T23:33:38.217Z" },
+    { url = "https://files.pythonhosted.org/packages/15/a7/9bc1cd626d7bf6869bfedf27b91b6ab5dd607758bf8e959d6fa80c6a59cb/numpy-2.4.6-cp311-cp311-win32.whl", hash = "sha256:ddea102b48f9e339f3948bf22040944184627a30fdf7f858667673b9c5f033c8", size = 6233945, upload-time = "2026-05-18T23:33:41.331Z" },
+    { url = "https://files.pythonhosted.org/packages/c5/31/7fc6239c12bce7e931463251cca4426c465e1876ba3cc785402ef4dd8f4e/numpy-2.4.6-cp311-cp311-win_amd64.whl", hash = "sha256:1e254a00cdf42b1e4d5b3d68d33af63268d41340d8885df2ab6470f2e1500147", size = 12608406, upload-time = "2026-05-18T23:33:44.131Z" },
+    { url = "https://files.pythonhosted.org/packages/27/83/140f85a466595a16382996a1bf06b2b54bcd597488921b0c9daaeeda72af/numpy-2.4.6-cp311-cp311-win_arm64.whl", hash = "sha256:ed9749eef4cbd126da3dc1d6bcb3a57f5eb7ac6a6484146bdbf743f552dfc577", size = 10479528, upload-time = "2026-05-18T23:33:50.725Z" },
+    { url = "https://files.pythonhosted.org/packages/95/2a/3d7b5ac8aac24feaf9ad7ed58f45b0bbc06d37e4338ae84c9f2298b570f9/numpy-2.4.6-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:001fbb8e08d942dd57599e781f2472269ee7f2755fae407b4f67b2f0b17da3f1", size = 16689119, upload-time = "2026-05-18T23:33:54.065Z" },
+    { url = "https://files.pythonhosted.org/packages/ea/12/92c4c131527599e8288d6918e888d88726f84d805d784b771f32408aeaef/numpy-2.4.6-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:ebfb099f8dcf083deef3ac1ca4c1503f387cf76296fcb3816b66f5ecb5f54fdb", size = 14699246, upload-time = "2026-05-18T23:33:57.621Z" },
+    { url = "https://files.pythonhosted.org/packages/ad/fe/c0a6b7b2ca128a8fb228575147073b660656734b8ebe4d76c8fd748dcc79/numpy-2.4.6-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:3213d622a0283a39a93d188f3cf72b26862df52fbb4ca3697f51705016523d41", size = 5204410, upload-time = "2026-05-18T23:34:00.302Z" },
+    { url = "https://files.pythonhosted.org/packages/f3/d4/9770d14ba719432bb90a421bfd443872ed0f70f7264b64bec12ea363d5fd/numpy-2.4.6-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:357cc07a6d7b0b182ff02249616a03742827ebb1277546b5c7cd7f7620a45698", size = 6551240, upload-time = "2026-05-18T23:34:02.852Z" },
+    { url = "https://files.pythonhosted.org/packages/c9/c6/50a46a6205feba2343f1d6d17438107c5dc491ed1c736e6ea68689fd906b/numpy-2.4.6-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5f9fb9157b4ce2971008323afe46053787b526ef624fea915b261468a8421a0f", size = 15671012, upload-time = "2026-05-18T23:34:05.485Z" },
+    { url = "https://files.pythonhosted.org/packages/99/60/14115e6364fa676c5397c2ad3004e527e9aa487abf5d0706ec81bbd08529/numpy-2.4.6-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:90f9849678c75fe7afa2d348ac842c168b0a4d3d61919687216dfc547976d853", size = 16645538, upload-time = "2026-05-18T23:34:09.265Z" },
+    { url = "https://files.pythonhosted.org/packages/ae/c5/693cbe59e57db94d2231fa519ca3978dc9e19da5a8f088588f5c6e947ff2/numpy-2.4.6-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:c1a2af6c6ef86344a6b0db6b97834208bf598db514f2b155042439b62605601a", size = 17020706, upload-time = "2026-05-18T23:34:13.053Z" },
+    { url = "https://files.pythonhosted.org/packages/ef/fc/85b7c4eff9b4966ade25c2273cf7e7012e92366c032058653934b37de044/numpy-2.4.6-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:e5805d5a22fd19c8ccff10a9561f9df94436b0545619ea579db2d3c35294bce2", size = 18368541, upload-time = "2026-05-18T23:34:17.024Z" },
+    { url = "https://files.pythonhosted.org/packages/f6/81/e1b27545deedce7f4a0b348618c6b62d74e36a4dc9ccd42f3eb2f85eee32/numpy-2.4.6-cp312-cp312-win32.whl", hash = "sha256:e3eeb0aabd6bd5ce64faae67e9935203a6991b4bc2a485a767fbafb2c5125f45", size = 5962825, upload-time = "2026-05-18T23:34:20.3Z" },
+    { url = "https://files.pythonhosted.org/packages/ab/ca/feab00bd44aa5fe1ad2c18f08b4d3bb92e26484b0b1d1443897809ed528c/numpy-2.4.6-cp312-cp312-win_amd64.whl", hash = "sha256:d8e8286dd7cea7895157318d1b91cdacac64c479f3cbc8dce548331728484751", size = 12321687, upload-time = "2026-05-18T23:34:23.095Z" },
+    { url = "https://files.pythonhosted.org/packages/63/cf/5a6d34850a39d1093558564f77ee8e8e0bee5061151b8f05a55711001ec7/numpy-2.4.6-cp312-cp312-win_arm64.whl", hash = "sha256:4081eb135ac24158bd51cdfbef16f1c64df7063b1143f24731387137c092bec8", size = 10221482, upload-time = "2026-05-18T23:34:25.876Z" },
+    { url = "https://files.pythonhosted.org/packages/fb/82/bdab26d7438c6791ca31b7c024ca37c1eab8b726ba236129005cd4a06e45/numpy-2.4.6-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:511dbaf848decaaaf4b4ca48032619fb3138710c4bf7da7617765edad1ef96b0", size = 16684648, upload-time = "2026-05-18T23:34:29.41Z" },
+    { url = "https://files.pythonhosted.org/packages/1b/30/a80189bcc7f5e4258b3fbc3968d909d1756f54d023299ecc39ad6fdb9ef8/numpy-2.4.6-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:bf162abab1c1a736333192707cef898e735a5ca00f38f27eeedf44b39d9e85eb", size = 14693902, upload-time = "2026-05-18T23:34:33.013Z" },
+    { url = "https://files.pythonhosted.org/packages/97/12/70b5d0d7c15e1ebb8a6a84a8caa1d19e181d84fb58bb6d70aca29099dec1/numpy-2.4.6-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:043191bfa8eab18c776647b62723ac9dddece59743b13f49b2016094129c2b3f", size = 5198992, upload-time = "2026-05-18T23:34:36.132Z" },
+    { url = "https://files.pythonhosted.org/packages/ba/8c/ebd2a8f8a83541f8d38cc5667e8c2b69cecfd30da6e45693e8158857d44b/numpy-2.4.6-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:6180d8b35af935aed8ece3a85e0a43f87393ae0ac87c8d2c8bd2c993f7270ef3", size = 6546944, upload-time = "2026-05-18T23:34:38.484Z" },
+    { url = "https://files.pythonhosted.org/packages/bb/c5/7b863a97a91671a0338f4253bd3b5a3d3852f0692dae91711c9f4a10e787/numpy-2.4.6-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:72fbe16c6fac95aedf5937fa873445cec2110be35d8a4e9433d7501fd98dae6b", size = 15669392, upload-time = "2026-05-18T23:34:41.257Z" },
+    { url = "https://files.pythonhosted.org/packages/a5/9d/3584b9984ca4c047aea75214ce1a4c4c73d849bd71b604264b7f5653f8a8/numpy-2.4.6-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a7830bab239b79cda9c08c2da014761cafb48da6150e1da17ac06283f43b6089", size = 16633220, upload-time = "2026-05-18T23:34:45.075Z" },
+    { url = "https://files.pythonhosted.org/packages/05/ae/7c67fba23bd98caec7c99261f3a16072ade14813486b0282cb29846de832/numpy-2.4.6-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:ef4aea96ce4d3b074422cb4f2f64e216bf9e213004bb58ecfdf50ea02ea8eb9a", size = 17020800, upload-time = "2026-05-18T23:34:49.065Z" },
+    { url = "https://files.pythonhosted.org/packages/d9/5d/3b6725cb31d983c5e66916f5d36f6d7e5521129e4c4404d64f918292a5b6/numpy-2.4.6-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:dfa20cc6ca228e6b155b11da03825975ce66aea520985dbbddf0f2a5a495c605", size = 18357600, upload-time = "2026-05-18T23:34:52.709Z" },
+    { url = "https://files.pythonhosted.org/packages/f7/da/2ccc6c2fe8898dee01d90c75c5f5f914a23daf99e3e0f59516a08760c8b5/numpy-2.4.6-cp313-cp313-win32.whl", hash = "sha256:56b39e5e0622a09a25bf5baf62f4bcf0cb8a41ae6e2819cf49bbc5a74c083f91", size = 5961134, upload-time = "2026-05-18T23:34:55.618Z" },
+    { url = "https://files.pythonhosted.org/packages/b5/cd/9cc4dc876fb065d5c220aae4d5e14826b2715331bb7618ce1fb07a679d99/numpy-2.4.6-cp313-cp313-win_amd64.whl", hash = "sha256:c4fc99836233ea196540b17ab0983aff60ed07941751930f5f4d05bc3b3b7359", size = 12318598, upload-time = "2026-05-18T23:34:58.928Z" },
+    { url = "https://files.pythonhosted.org/packages/39/1e/c0bcba1f8694116485fe28fd1be698c278fcda4141c5b0e53a2aed8b12a8/numpy-2.4.6-cp313-cp313-win_arm64.whl", hash = "sha256:a7c711e21628b52034bb5ab8d1bce291f752fcc5e92accc615778acee1ff4778", size = 10222272, upload-time = "2026-05-18T23:35:02.167Z" },
+    { url = "https://files.pythonhosted.org/packages/63/6d/cc5619247c8f4204e507f5883528372e4ac4bb189e579fb859a12e480b1f/numpy-2.4.6-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:112b06a867b235ef466ed3508ddf0238050df9c727cafb5301ac385b899189a1", size = 14821197, upload-time = "2026-05-18T23:35:05.468Z" },
+    { url = "https://files.pythonhosted.org/packages/00/58/f1c39161c87d9e9bed660f1ed4bafc0e403d5ec9650b6dd77aead07d489b/numpy-2.4.6-cp313-cp313t-macosx_14_0_arm64.whl", hash = "sha256:eaf7fa2de5c0be8ae6ff8e9bea2ccd725e980541244521d8d4b5f3354a27babe", size = 5326287, upload-time = "2026-05-18T23:35:08.693Z" },
+    { url = "https://files.pythonhosted.org/packages/af/57/3917ab0fd97f271a8694513581b8a36c655f111c446852c302f04ccdb6fc/numpy-2.4.6-cp313-cp313t-macosx_14_0_x86_64.whl", hash = "sha256:7265a2f3d436e54ef9f2b52b5c937e6be778781bd97a590319d7348f1c1ca997", size = 6646763, upload-time = "2026-05-18T23:35:11.459Z" },
+    { url = "https://files.pythonhosted.org/packages/eb/0f/037e64c494b67581ae18193d770adef354c41f3f2c8ebf865602d949bf8f/numpy-2.4.6-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f74a575920ab21fe304421a3fc28793d82e299cae9eccb37084e9fc7f3617c20", size = 15728070, upload-time = "2026-05-18T23:35:14.79Z" },
+    { url = "https://files.pythonhosted.org/packages/21/a6/5d2bae9c9542eb4df16dc9c46dc79c186e9bad53805dfa5399a6023c6db0/numpy-2.4.6-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ede83e07a75dd06bc501566c1eca2afc0d61677c1472ac9ad93fdee6e638a48d", size = 16681752, upload-time = "2026-05-18T23:35:18.836Z" },
+    { url = "https://files.pythonhosted.org/packages/92/14/23d1dfb410ae362cd59ce53e936b1513d545eb40db3949ced632e19a459e/numpy-2.4.6-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:68bb27509ac1b9a3443094260f6326150663b06abe40b73a2f81160623da5b67", size = 17086024, upload-time = "2026-05-18T23:35:22.52Z" },
+    { url = "https://files.pythonhosted.org/packages/4b/6e/23595a2c642cdf3bc567877064bdd7f91c8b0038a4453cf2daf7248eafe9/numpy-2.4.6-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:a0df0043bdb289bde1f62da130d20df23d58b45429f752bc7a8fc5325a225ecd", size = 18403398, upload-time = "2026-05-18T23:35:26.398Z" },
+    { url = "https://files.pythonhosted.org/packages/8a/90/0ac3bc947217e66dec77e7cbc6a1979d1af70b6461b82f620d3bccd5e4c8/numpy-2.4.6-cp313-cp313t-win32.whl", hash = "sha256:29a287e0cf63ff528da061de6b9f64a4618da591ca1046aafc54062e40ca7eab", size = 6084971, upload-time = "2026-05-18T23:35:29.387Z" },
+    { url = "https://files.pythonhosted.org/packages/77/71/5673e351671a1d2bd6063b91b44f70c0affea7d1516fa7a6572941ba4aa1/numpy-2.4.6-cp313-cp313t-win_amd64.whl", hash = "sha256:25c692919ac5a01f170a3bfcd62d745b24fd095c353d50812637d6fcab442e75", size = 12458532, upload-time = "2026-05-18T23:35:32.175Z" },
+    { url = "https://files.pythonhosted.org/packages/3f/88/19d3503c5046e688f049274b27a3ef3d771152fa80d3ba3d01a3dff61abe/numpy-2.4.6-cp313-cp313t-win_arm64.whl", hash = "sha256:1e978ec1e8bd0e0e4de6bb75de9d30cbb74db6b6a2bb727618613703ca0167dd", size = 10291881, upload-time = "2026-05-18T23:35:35.465Z" },
+    { url = "https://files.pythonhosted.org/packages/f8/91/3ab2044d05fd16d343c5ac2e69b127f1b2854040dd20b193257c78028bd3/numpy-2.4.6-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:06ca2f61ec4385a07a6977c55ba998a4466c123642b4a32694d3128fce18c079", size = 16683458, upload-time = "2026-05-18T23:35:38.353Z" },
+    { url = "https://files.pythonhosted.org/packages/8e/62/764ce66fa4147ae6d73071a3abf804ffe606f174618697c571acdf26a7c9/numpy-2.4.6-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:38efbc8de75c7a0fc1ac190162d892787f3f47b57cc291231aafee36b80982b7", size = 14704559, upload-time = "2026-05-18T23:35:42.14Z" },
+    { url = "https://files.pythonhosted.org/packages/60/61/23f27c172f022e04025b7dc2367f4d63c1a398120607ec896228649a6f48/numpy-2.4.6-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:d581b735e177fdcdce6fed8e7e8880a3fb6ee4e3653a3ac6af01c6f4c03effc5", size = 5209716, upload-time = "2026-05-18T23:35:45.377Z" },
+    { url = "https://files.pythonhosted.org/packages/03/71/21cf70dc6ea3e3acb95fc53a265b2fc248b981f0194ceb5b475271b8809d/numpy-2.4.6-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:0a041d3d761dc3c35cc56ce0351506a02bcbc25f7b169f652435141a17db9096", size = 6543947, upload-time = "2026-05-18T23:35:47.926Z" },
+    { url = "https://files.pythonhosted.org/packages/d5/91/64288395ee1799bd2e0b04a305dce9666da90c961e1f3fe982a05ee1c036/numpy-2.4.6-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:40fdc1ae7125e518ea98e53e69a4ebc27e1fd50510c47b7ea130cf21e5e1d42b", size = 15685197, upload-time = "2026-05-18T23:35:50.863Z" },
+    { url = "https://files.pythonhosted.org/packages/f3/eb/ebffaa97dc55502df69584a8f0dcf07f69a3e0b3e2323670a2722db9aa39/numpy-2.4.6-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a2c306dea656c12c68f51f4cea133cbe78ca7435eb28c735eac1d3ebe73be6e8", size = 16638245, upload-time = "2026-05-18T23:35:54.752Z" },
+    { url = "https://files.pythonhosted.org/packages/b8/0b/54f9da33128d7e350fab89c7455902eeae70349ee52bddb448dc4a576f45/numpy-2.4.6-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:33111801a01c12a8a1e3721f0a9232f8cfc8ae2c6b7098167e6f623c6073f402", size = 17036587, upload-time = "2026-05-18T23:35:58.355Z" },
+    { url = "https://files.pythonhosted.org/packages/b6/f0/fdebc1052db1cc37c64beb22072d67cd6d1c71adca1299f53dec2b5e20d3/numpy-2.4.6-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ae506e6902902557576a26ff33eda8695e7ecb3cb36c3b573a0765dee114ebdb", size = 18363226, upload-time = "2026-05-18T23:36:02.845Z" },
+    { url = "https://files.pythonhosted.org/packages/aa/b4/298628d98c72b57e57f7165ae6a481a1deaf6f3c28262a6e4c739c275930/numpy-2.4.6-cp314-cp314-win32.whl", hash = "sha256:aaf159caa35993cb1f56fb9b8e4610d35758e7ca005412eb1daa856a78c9c4b1", size = 6010196, upload-time = "2026-05-18T23:36:05.92Z" },
+    { url = "https://files.pythonhosted.org/packages/df/ac/46de6dda46478f7942f839e094970be2d4a861e005c4b3bf07c92e291a09/numpy-2.4.6-cp314-cp314-win_amd64.whl", hash = "sha256:b507f5c4c1d508876d1819b6bf9a49d365b96320b5d4993426b33a23ca4b8261", size = 12450334, upload-time = "2026-05-18T23:36:09.107Z" },
+    { url = "https://files.pythonhosted.org/packages/78/92/b8b798ac784102c0da830d2257d59358e3d3d90d1e2b3f2575dad976c5cf/numpy-2.4.6-cp314-cp314-win_arm64.whl", hash = "sha256:6f41ae150c4e32db4f3310cdaf64b1593a03dbabe29eec77fc9b50fe64061df6", size = 10495678, upload-time = "2026-05-18T23:36:12.766Z" },
+    { url = "https://files.pythonhosted.org/packages/30/34/ec28d1aa8115971537c01469ab2011ee96827930f0a124de1000cc2a7ed7/numpy-2.4.6-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ece3d2cfe132e7d51f44a832b303895e6f2d499c5e74dfbdb06ee246147a304a", size = 14823672, upload-time = "2026-05-18T23:36:16.473Z" },
+    { url = "https://files.pythonhosted.org/packages/16/bd/f6d1fede4e54e8042a7ff97bb495510f3c220f94bcd9e8b228e87c92cc0d/numpy-2.4.6-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:e3e5193ef5a3dc73bceee50f7fdc2c90dbb76c42df8d8fae3d1067a583df579e", size = 5328731, upload-time = "2026-05-18T23:36:19.767Z" },
+    { url = "https://files.pythonhosted.org/packages/f4/f0/e105b9e2fd728a9910103884decd6951d9dd73896b914a98d9a231de02ee/numpy-2.4.6-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:17f9ade344e7d9b464a084d69bcf18fc691cb1db67c62ed80820bf4926d78f0e", size = 6649805, upload-time = "2026-05-18T23:36:22.266Z" },
+    { url = "https://files.pythonhosted.org/packages/82/dd/1206a7ca6ab15e3f02069707ca96222e202af681bb73756da7527f3cb837/numpy-2.4.6-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9cd5ffd25db4e7ba6a375693b3fc0fc1791ec636c17db3720da19bde7180ec43", size = 15730496, upload-time = "2026-05-18T23:36:25.713Z" },
+    { url = "https://files.pythonhosted.org/packages/51/e7/38d3ea825dcab85a591734decb2f6c67caa7c8367d374df1a1c3842f9b07/numpy-2.4.6-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7d92c3819208a60205a12a245c91ad70cb0a85336659b19b834205573ac8456e", size = 16679616, upload-time = "2026-05-18T23:36:29.652Z" },
+    { url = "https://files.pythonhosted.org/packages/93/b7/caabfdf53edf663e0b4eb74d7d405d83baef09eb5e83bcd32d601d72b93e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e85b752a1e912b70eaad4fafbd4d1238007ab221de2009b9a2f5ae7461239895", size = 17085145, upload-time = "2026-05-18T23:36:33.449Z" },
+    { url = "https://files.pythonhosted.org/packages/f9/45/68d7c33a6bcf3e5aa3bdbd57a367e6f615286dfd6482f97e8ffeb734306e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:29cb7f67d10b479ff07c17d33e39f78c07f71c40ef30d63c153d340e96cd3fb4", size = 18403813, upload-time = "2026-05-18T23:36:37.369Z" },
+    { url = "https://files.pythonhosted.org/packages/9c/50/0753655aa844c99cd9e018aacf76f130f1bd81d881bb74bc0aef5d73a8ba/numpy-2.4.6-cp314-cp314t-win32.whl", hash = "sha256:260a5d70215b61ab4fadf5c7baacd64821842975eea312125ed3c39a6391b063", size = 6156982, upload-time = "2026-05-18T23:36:40.817Z" },
+    { url = "https://files.pythonhosted.org/packages/b2/d4/7c67becf668f973cb490cec3e98dfd799d866f9c989a54d355672cfa0db6/numpy-2.4.6-cp314-cp314t-win_amd64.whl", hash = "sha256:81a1cca95ed5bb92aa8b10dd2cdc9a0d3853a50fad926c28b5d7e8ea54389627", size = 12638908, upload-time = "2026-05-18T23:36:43.996Z" },
+    { url = "https://files.pythonhosted.org/packages/43/bb/e1c71a4295b1b1d1393d50dbb4f2a36283c6859d9d3892e84f00ec5a91d5/numpy-2.4.6-cp314-cp314t-win_arm64.whl", hash = "sha256:0c9136e14ed34a9e343a31c533d78a9813a69a3148332bce5e9821cb2f996e66", size = 10565867, upload-time = "2026-05-18T23:36:47.114Z" },
+    { url = "https://files.pythonhosted.org/packages/de/12/b422cc84439adc0d00de605bf4a308890ae5c26f2c71fbd73e5d08fbb0dd/numpy-2.4.6-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:55cced7c52e981362f708ad635198e97a752dfba412cc03c23bbf3bd8d5cd662", size = 16847511, upload-time = "2026-05-18T23:36:50.673Z" },
+    { url = "https://files.pythonhosted.org/packages/44/53/f481bef68011740f8849418d82db07230e825013f31f4eef5ba5b805316a/numpy-2.4.6-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:d6da64deb6b8ed903e7560180a92f2d804ee1ba5eeb849ac2748b8c1aba1f6d7", size = 14889064, upload-time = "2026-05-18T23:36:53.879Z" },
+    { url = "https://files.pythonhosted.org/packages/7f/57/42ed575c10ced8af951d426bc4e1f8aff16fd851db33f067036215a7f860/numpy-2.4.6-pp311-pypy311_pp73-macosx_14_0_arm64.whl", hash = "sha256:68a5124b13fa6cc2086764a20005d30bc0548146f7f5322f02fce212ca14317f", size = 5394157, upload-time = "2026-05-18T23:36:57.194Z" },
+    { url = "https://files.pythonhosted.org/packages/6a/ef/f66cc724fcc36c1e364c67f51ae9146090b8b584f27d58b97fdae3edd737/numpy-2.4.6-pp311-pypy311_pp73-macosx_14_0_x86_64.whl", hash = "sha256:948424b06129ce883307e8cff868c31396d8dc7630a59c61d70d98dbe70f222c", size = 6708728, upload-time = "2026-05-18T23:36:59.575Z" },
+    { url = "https://files.pythonhosted.org/packages/1a/9c/c531f2293b91265d8b48e9b329f54fdd7ffae73cb4134ea10cca4237e9cc/numpy-2.4.6-pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5dbbdb29840ca3d91ee0fece42fc29278886d908280bfec0a5846c6f901a3eb0", size = 15798374, upload-time = "2026-05-18T23:37:02.674Z" },
+    { url = "https://files.pythonhosted.org/packages/1a/b0/413077f6b1153ed3cba361401c6783bbad6114804a000cc22eb71c13e190/numpy-2.4.6-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8ad03c0965fb3c692200e74d458ca28c1dbb4ce96f9a479a8aa041ad5fabca02", size = 16747286, upload-time = "2026-05-18T23:37:06.327Z" },
+    { url = "https://files.pythonhosted.org/packages/15/ce/e5ec180bc41812edcd8daeb8639d205622c0e8c02259d8ab25a0201b3c2a/numpy-2.4.6-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:2803abfebfc990042cd494d8ce2d5f82e9d847af6d35ec486923aa19dbad5e73", size = 12504263, upload-time = "2026-05-18T23:37:09.715Z" },
+]
+
+[[package]]
+name = "packaging"
+version = "26.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/d7/f1/e7a6dd94a8d4a5626c03e4e99c87f241ba9e350cd9e6d75123f992427270/packaging-26.2.tar.gz", hash = "sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661", size = 228134, upload-time = "2026-04-24T20:15:23.917Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e", size = 100195, upload-time = "2026-04-24T20:15:22.081Z" },
+]
+
+[[package]]
+name = "pluggy"
+version = "1.6.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },
+]
+
+[[package]]
+name = "pygments"
+version = "2.20.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" },
+]
+
+[[package]]
+name = "pytest"
+version = "9.0.3"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "colorama", marker = "sys_platform == 'win32'" },
+    { name = "exceptiongroup", marker = "python_full_version < '3.11'" },
+    { name = "iniconfig" },
+    { name = "packaging" },
+    { name = "pluggy" },
+    { name = "pygments" },
+    { name = "tomli", marker = "python_full_version < '3.11'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/7d/0d/549bd94f1a0a402dc8cf64563a117c0f3765662e2e668477624baeec44d5/pytest-9.0.3.tar.gz", hash = "sha256:b86ada508af81d19edeb213c681b1d48246c1a91d304c6c81a427674c17eb91c", size = 1572165, upload-time = "2026-04-07T17:16:18.027Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl", hash = "sha256:2c5efc453d45394fdd706ade797c0a81091eccd1d6e4bccfcd476e2b8e0ab5d9", size = 375249, upload-time = "2026-04-07T17:16:16.13Z" },
+]
+
+[[package]]
+name = "tomli"
+version = "2.4.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/22/de/48c59722572767841493b26183a0d1cc411d54fd759c5607c4590b6563a6/tomli-2.4.1.tar.gz", hash = "sha256:7c7e1a961a0b2f2472c1ac5b69affa0ae1132c39adcb67aba98568702b9cc23f", size = 17543, upload-time = "2026-03-25T20:22:03.828Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/f4/11/db3d5885d8528263d8adc260bb2d28ebf1270b96e98f0e0268d32b8d9900/tomli-2.4.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:f8f0fc26ec2cc2b965b7a3b87cd19c5c6b8c5e5f436b984e85f486d652285c30", size = 154704, upload-time = "2026-03-25T20:21:10.473Z" },
+    { url = "https://files.pythonhosted.org/packages/6d/f7/675db52c7e46064a9aa928885a9b20f4124ecb9bc2e1ce74c9106648d202/tomli-2.4.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:4ab97e64ccda8756376892c53a72bd1f964e519c77236368527f758fbc36a53a", size = 149454, upload-time = "2026-03-25T20:21:12.036Z" },
+    { url = "https://files.pythonhosted.org/packages/61/71/81c50943cf953efa35bce7646caab3cf457a7d8c030b27cfb40d7235f9ee/tomli-2.4.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:96481a5786729fd470164b47cdb3e0e58062a496f455ee41b4403be77cb5a076", size = 237561, upload-time = "2026-03-25T20:21:13.098Z" },
+    { url = "https://files.pythonhosted.org/packages/48/c1/f41d9cb618acccca7df82aaf682f9b49013c9397212cb9f53219e3abac37/tomli-2.4.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5a881ab208c0baf688221f8cecc5401bd291d67e38a1ac884d6736cbcd8247e9", size = 243824, upload-time = "2026-03-25T20:21:14.569Z" },
+    { url = "https://files.pythonhosted.org/packages/22/e4/5a816ecdd1f8ca51fb756ef684b90f2780afc52fc67f987e3c61d800a46d/tomli-2.4.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:47149d5bd38761ac8be13a84864bf0b7b70bc051806bc3669ab1cbc56216b23c", size = 242227, upload-time = "2026-03-25T20:21:15.712Z" },
+    { url = "https://files.pythonhosted.org/packages/6b/49/2b2a0ef529aa6eec245d25f0c703e020a73955ad7edf73e7f54ddc608aa5/tomli-2.4.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:ec9bfaf3ad2df51ace80688143a6a4ebc09a248f6ff781a9945e51937008fcbc", size = 247859, upload-time = "2026-03-25T20:21:17.001Z" },
+    { url = "https://files.pythonhosted.org/packages/83/bd/6c1a630eaca337e1e78c5903104f831bda934c426f9231429396ce3c3467/tomli-2.4.1-cp311-cp311-win32.whl", hash = "sha256:ff2983983d34813c1aeb0fa89091e76c3a22889ee83ab27c5eeb45100560c049", size = 97204, upload-time = "2026-03-25T20:21:18.079Z" },
+    { url = "https://files.pythonhosted.org/packages/42/59/71461df1a885647e10b6bb7802d0b8e66480c61f3f43079e0dcd315b3954/tomli-2.4.1-cp311-cp311-win_amd64.whl", hash = "sha256:5ee18d9ebdb417e384b58fe414e8d6af9f4e7a0ae761519fb50f721de398dd4e", size = 108084, upload-time = "2026-03-25T20:21:18.978Z" },
+    { url = "https://files.pythonhosted.org/packages/b8/83/dceca96142499c069475b790e7913b1044c1a4337e700751f48ed723f883/tomli-2.4.1-cp311-cp311-win_arm64.whl", hash = "sha256:c2541745709bad0264b7d4705ad453b76ccd191e64aa6f0fc66b69a293a45ece", size = 95285, upload-time = "2026-03-25T20:21:20.309Z" },
+    { url = "https://files.pythonhosted.org/packages/c1/ba/42f134a3fe2b370f555f44b1d72feebb94debcab01676bf918d0cb70e9aa/tomli-2.4.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:c742f741d58a28940ce01d58f0ab2ea3ced8b12402f162f4d534dfe18ba1cd6a", size = 155924, upload-time = "2026-03-25T20:21:21.626Z" },
+    { url = "https://files.pythonhosted.org/packages/dc/c7/62d7a17c26487ade21c5422b646110f2162f1fcc95980ef7f63e73c68f14/tomli-2.4.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:7f86fd587c4ed9dd76f318225e7d9b29cfc5a9d43de44e5754db8d1128487085", size = 150018, upload-time = "2026-03-25T20:21:23.002Z" },
+    { url = "https://files.pythonhosted.org/packages/5c/05/79d13d7c15f13bdef410bdd49a6485b1c37d28968314eabee452c22a7fda/tomli-2.4.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ff18e6a727ee0ab0388507b89d1bc6a22b138d1e2fa56d1ad494586d61d2eae9", size = 244948, upload-time = "2026-03-25T20:21:24.04Z" },
+    { url = "https://files.pythonhosted.org/packages/10/90/d62ce007a1c80d0b2c93e02cab211224756240884751b94ca72df8a875ca/tomli-2.4.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:136443dbd7e1dee43c68ac2694fde36b2849865fa258d39bf822c10e8068eac5", size = 253341, upload-time = "2026-03-25T20:21:25.177Z" },
+    { url = "https://files.pythonhosted.org/packages/1a/7e/caf6496d60152ad4ed09282c1885cca4eea150bfd007da84aea07bcc0a3e/tomli-2.4.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:5e262d41726bc187e69af7825504c933b6794dc3fbd5945e41a79bb14c31f585", size = 248159, upload-time = "2026-03-25T20:21:26.364Z" },
+    { url = "https://files.pythonhosted.org/packages/99/e7/c6f69c3120de34bbd882c6fba7975f3d7a746e9218e56ab46a1bc4b42552/tomli-2.4.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:5cb41aa38891e073ee49d55fbc7839cfdb2bc0e600add13874d048c94aadddd1", size = 253290, upload-time = "2026-03-25T20:21:27.46Z" },
+    { url = "https://files.pythonhosted.org/packages/d6/2f/4a3c322f22c5c66c4b836ec58211641a4067364f5dcdd7b974b4c5da300c/tomli-2.4.1-cp312-cp312-win32.whl", hash = "sha256:da25dc3563bff5965356133435b757a795a17b17d01dbc0f42fb32447ddfd917", size = 98141, upload-time = "2026-03-25T20:21:28.492Z" },
+    { url = "https://files.pythonhosted.org/packages/24/22/4daacd05391b92c55759d55eaee21e1dfaea86ce5c571f10083360adf534/tomli-2.4.1-cp312-cp312-win_amd64.whl", hash = "sha256:52c8ef851d9a240f11a88c003eacb03c31fc1c9c4ec64a99a0f922b93874fda9", size = 108847, upload-time = "2026-03-25T20:21:29.386Z" },
+    { url = "https://files.pythonhosted.org/packages/68/fd/70e768887666ddd9e9f5d85129e84910f2db2796f9096aa02b721a53098d/tomli-2.4.1-cp312-cp312-win_arm64.whl", hash = "sha256:f758f1b9299d059cc3f6546ae2af89670cb1c4d48ea29c3cacc4fe7de3058257", size = 95088, upload-time = "2026-03-25T20:21:30.677Z" },
+    { url = "https://files.pythonhosted.org/packages/07/06/b823a7e818c756d9a7123ba2cda7d07bc2dd32835648d1a7b7b7a05d848d/tomli-2.4.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:36d2bd2ad5fb9eaddba5226aa02c8ec3fa4f192631e347b3ed28186d43be6b54", size = 155866, upload-time = "2026-03-25T20:21:31.65Z" },
+    { url = "https://files.pythonhosted.org/packages/14/6f/12645cf7f08e1a20c7eb8c297c6f11d31c1b50f316a7e7e1e1de6e2e7b7e/tomli-2.4.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:eb0dc4e38e6a1fd579e5d50369aa2e10acfc9cace504579b2faabb478e76941a", size = 149887, upload-time = "2026-03-25T20:21:33.028Z" },
+    { url = "https://files.pythonhosted.org/packages/5c/e0/90637574e5e7212c09099c67ad349b04ec4d6020324539297b634a0192b0/tomli-2.4.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c7f2c7f2b9ca6bdeef8f0fa897f8e05085923eb091721675170254cbc5b02897", size = 243704, upload-time = "2026-03-25T20:21:34.51Z" },
+    { url = "https://files.pythonhosted.org/packages/10/8f/d3ddb16c5a4befdf31a23307f72828686ab2096f068eaf56631e136c1fdd/tomli-2.4.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f3c6818a1a86dd6dca7ddcaaf76947d5ba31aecc28cb1b67009a5877c9a64f3f", size = 251628, upload-time = "2026-03-25T20:21:36.012Z" },
+    { url = "https://files.pythonhosted.org/packages/e3/f1/dbeeb9116715abee2485bf0a12d07a8f31af94d71608c171c45f64c0469d/tomli-2.4.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:d312ef37c91508b0ab2cee7da26ec0b3ed2f03ce12bd87a588d771ae15dcf82d", size = 247180, upload-time = "2026-03-25T20:21:37.136Z" },
+    { url = "https://files.pythonhosted.org/packages/d3/74/16336ffd19ed4da28a70959f92f506233bd7cfc2332b20bdb01591e8b1d1/tomli-2.4.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:51529d40e3ca50046d7606fa99ce3956a617f9b36380da3b7f0dd3dd28e68cb5", size = 251674, upload-time = "2026-03-25T20:21:38.298Z" },
+    { url = "https://files.pythonhosted.org/packages/16/f9/229fa3434c590ddf6c0aa9af64d3af4b752540686cace29e6281e3458469/tomli-2.4.1-cp313-cp313-win32.whl", hash = "sha256:2190f2e9dd7508d2a90ded5ed369255980a1bcdd58e52f7fe24b8162bf9fedbd", size = 97976, upload-time = "2026-03-25T20:21:39.316Z" },
+    { url = "https://files.pythonhosted.org/packages/6a/1e/71dfd96bcc1c775420cb8befe7a9d35f2e5b1309798f009dca17b7708c1e/tomli-2.4.1-cp313-cp313-win_amd64.whl", hash = "sha256:8d65a2fbf9d2f8352685bc1364177ee3923d6baf5e7f43ea4959d7d8bc326a36", size = 108755, upload-time = "2026-03-25T20:21:40.248Z" },
+    { url = "https://files.pythonhosted.org/packages/83/7a/d34f422a021d62420b78f5c538e5b102f62bea616d1d75a13f0a88acb04a/tomli-2.4.1-cp313-cp313-win_arm64.whl", hash = "sha256:4b605484e43cdc43f0954ddae319fb75f04cc10dd80d830540060ee7cd0243cd", size = 95265, upload-time = "2026-03-25T20:21:41.219Z" },
+    { url = "https://files.pythonhosted.org/packages/3c/fb/9a5c8d27dbab540869f7c1f8eb0abb3244189ce780ba9cd73f3770662072/tomli-2.4.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:fd0409a3653af6c147209d267a0e4243f0ae46b011aa978b1080359fddc9b6cf", size = 155726, upload-time = "2026-03-25T20:21:42.23Z" },
+    { url = "https://files.pythonhosted.org/packages/62/05/d2f816630cc771ad836af54f5001f47a6f611d2d39535364f148b6a92d6b/tomli-2.4.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:a120733b01c45e9a0c34aeef92bf0cf1d56cfe81ed9d47d562f9ed591a9828ac", size = 149859, upload-time = "2026-03-25T20:21:43.386Z" },
+    { url = "https://files.pythonhosted.org/packages/ce/48/66341bdb858ad9bd0ceab5a86f90eddab127cf8b046418009f2125630ecb/tomli-2.4.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:559db847dc486944896521f68d8190be1c9e719fced785720d2216fe7022b662", size = 244713, upload-time = "2026-03-25T20:21:44.474Z" },
+    { url = "https://files.pythonhosted.org/packages/df/6d/c5fad00d82b3c7a3ab6189bd4b10e60466f22cfe8a08a9394185c8a8111c/tomli-2.4.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:01f520d4f53ef97964a240a035ec2a869fe1a37dde002b57ebc4417a27ccd853", size = 252084, upload-time = "2026-03-25T20:21:45.62Z" },
+    { url = "https://files.pythonhosted.org/packages/00/71/3a69e86f3eafe8c7a59d008d245888051005bd657760e96d5fbfb0b740c2/tomli-2.4.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7f94b27a62cfad8496c8d2513e1a222dd446f095fca8987fceef261225538a15", size = 247973, upload-time = "2026-03-25T20:21:46.937Z" },
+    { url = "https://files.pythonhosted.org/packages/67/50/361e986652847fec4bd5e4a0208752fbe64689c603c7ae5ea7cb16b1c0ca/tomli-2.4.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ede3e6487c5ef5d28634ba3f31f989030ad6af71edfb0055cbbd14189ff240ba", size = 256223, upload-time = "2026-03-25T20:21:48.467Z" },
+    { url = "https://files.pythonhosted.org/packages/8c/9a/b4173689a9203472e5467217e0154b00e260621caa227b6fa01feab16998/tomli-2.4.1-cp314-cp314-win32.whl", hash = "sha256:3d48a93ee1c9b79c04bb38772ee1b64dcf18ff43085896ea460ca8dec96f35f6", size = 98973, upload-time = "2026-03-25T20:21:49.526Z" },
+    { url = "https://files.pythonhosted.org/packages/14/58/640ac93bf230cd27d002462c9af0d837779f8773bc03dee06b5835208214/tomli-2.4.1-cp314-cp314-win_amd64.whl", hash = "sha256:88dceee75c2c63af144e456745e10101eb67361050196b0b6af5d717254dddf7", size = 109082, upload-time = "2026-03-25T20:21:50.506Z" },
+    { url = "https://files.pythonhosted.org/packages/d5/2f/702d5e05b227401c1068f0d386d79a589bb12bf64c3d2c72ce0631e3bc49/tomli-2.4.1-cp314-cp314-win_arm64.whl", hash = "sha256:b8c198f8c1805dc42708689ed6864951fd2494f924149d3e4bce7710f8eb5232", size = 96490, upload-time = "2026-03-25T20:21:51.474Z" },
+    { url = "https://files.pythonhosted.org/packages/45/4b/b877b05c8ba62927d9865dd980e34a755de541eb65fffba52b4cc495d4d2/tomli-2.4.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:d4d8fe59808a54658fcc0160ecfb1b30f9089906c50b23bcb4c69eddc19ec2b4", size = 164263, upload-time = "2026-03-25T20:21:52.543Z" },
+    { url = "https://files.pythonhosted.org/packages/24/79/6ab420d37a270b89f7195dec5448f79400d9e9c1826df982f3f8e97b24fd/tomli-2.4.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7008df2e7655c495dd12d2a4ad038ff878d4ca4b81fccaf82b714e07eae4402c", size = 160736, upload-time = "2026-03-25T20:21:53.674Z" },
+    { url = "https://files.pythonhosted.org/packages/02/e0/3630057d8eb170310785723ed5adcdfb7d50cb7e6455f85ba8a3deed642b/tomli-2.4.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1d8591993e228b0c930c4bb0db464bdad97b3289fb981255d6c9a41aedc84b2d", size = 270717, upload-time = "2026-03-25T20:21:55.129Z" },
+    { url = "https://files.pythonhosted.org/packages/7a/b4/1613716072e544d1a7891f548d8f9ec6ce2faf42ca65acae01d76ea06bb0/tomli-2.4.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:734e20b57ba95624ecf1841e72b53f6e186355e216e5412de414e3c51e5e3c41", size = 278461, upload-time = "2026-03-25T20:21:56.228Z" },
+    { url = "https://files.pythonhosted.org/packages/05/38/30f541baf6a3f6df77b3df16b01ba319221389e2da59427e221ef417ac0c/tomli-2.4.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:8a650c2dbafa08d42e51ba0b62740dae4ecb9338eefa093aa5c78ceb546fcd5c", size = 274855, upload-time = "2026-03-25T20:21:57.653Z" },
+    { url = "https://files.pythonhosted.org/packages/77/a3/ec9dd4fd2c38e98de34223b995a3b34813e6bdadf86c75314c928350ed14/tomli-2.4.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:504aa796fe0569bb43171066009ead363de03675276d2d121ac1a4572397870f", size = 283144, upload-time = "2026-03-25T20:21:59.089Z" },
+    { url = "https://files.pythonhosted.org/packages/ef/be/605a6261cac79fba2ec0c9827e986e00323a1945700969b8ee0b30d85453/tomli-2.4.1-cp314-cp314t-win32.whl", hash = "sha256:b1d22e6e9387bf4739fbe23bfa80e93f6b0373a7f1b96c6227c32bef95a4d7a8", size = 108683, upload-time = "2026-03-25T20:22:00.214Z" },
+    { url = "https://files.pythonhosted.org/packages/12/64/da524626d3b9cc40c168a13da8335fe1c51be12c0a63685cc6db7308daae/tomli-2.4.1-cp314-cp314t-win_amd64.whl", hash = "sha256:2c1c351919aca02858f740c6d33adea0c5deea37f9ecca1cc1ef9e884a619d26", size = 121196, upload-time = "2026-03-25T20:22:01.169Z" },
+    { url = "https://files.pythonhosted.org/packages/5a/cd/e80b62269fc78fc36c9af5a6b89c835baa8af28ff5ad28c7028d60860320/tomli-2.4.1-cp314-cp314t-win_arm64.whl", hash = "sha256:eab21f45c7f66c13f2a9e0e1535309cee140182a9cdae1e041d02e47291e8396", size = 100393, upload-time = "2026-03-25T20:22:02.137Z" },
+    { url = "https://files.pythonhosted.org/packages/7b/61/cceae43728b7de99d9b847560c262873a1f6c98202171fd5ed62640b494b/tomli-2.4.1-py3-none-any.whl", hash = "sha256:0d85819802132122da43cb86656f8d1f8c6587d54ae7dcaf30e90533028b49fe", size = 14583, upload-time = "2026-03-25T20:22:03.012Z" },
+]
+
+[[package]]
+name = "typing-extensions"
+version = "4.15.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/72/94/1a15dd82efb362ac84269196e94cf00f187f7ed21c242792a923cdb1c61f/typing_extensions-4.15.0.tar.gz", hash = "sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466", size = 109391, upload-time = "2025-08-25T13:49:26.313Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548", size = 44614, upload-time = "2025-08-25T13:49:24.86Z" },
+]