Microbench 3 (connector_tax): infrastructure for KV connector substrate tax

Validates the elastic_migration_v2 finding that kv_role=kv_both adds
TTFT p90 +45% even when PD-sep never fires. Replicates under
single-instance, synthetic, open-loop workload to disambiguate
mechanism cost from 8-instance feedback amplification.

Configurations (8):
  plain, noop_connector, mooncake_{producer,consumer,both},
  nixl_both, lmcache_only, multi_mooncake_lmcache.

Pre-flight verification gates risky configs (kv_consumer needs dummy
bootstrap, multi-connector composition, NoOp custom class loading).

Workload: two-phase sweep
  Phase A: rate {0.5..32} req/s × shape (4096, 256), saturation criteria
  Phase B: ref_safe rate × cartesian (input ∈ {512,4k,32k}, output ∈ {64,256,1024})

Step-timing patch enriches vLLM's existing AGENTIC_STEP_LOG_PATH emit
with step_duration_us and build_meta_us — directly measures per-step
substrate cost, not just user-visible TTFT/TPOT.

run_all.sh runs as 5-stage barrier:
  0 pre-flight + apply patch
  1 Phase A all configs
  2 pick ref_safe / ref_load
  3 Phase B all configs
  4 revert patch + analyze + plot

Outputs aggregate.{json,csv}, MANIFEST.tsv, and 5 figures.
Estimated runtime: 4-5.5 hours on idle dash0 H20.

microbench/connector_tax/DESIGN.md

@@ -0,0 +1,664 @@
+# Microbench 3: KV Connector Substrate Tax  (revision 2)
+
+## Goal
+
+Validate the headline claim from
+`analysis/characterization/elastic_migration_v2/README.md` Result 1:
+
+> Switching the vLLM launch from plain to `kv_role=kv_both` without ever
+> triggering PD-sep already costs **TTFT p90 +45%, TPOT p90 +25%,
+> hotspot index +19%**.
+
+That claim was measured on 8 instances with a 1214-request real-trace
+replay under saturated coupling. We replicate it with **single-instance,
+synthetic, open-loop** workload so we can:
+
+1. Disambiguate **vLLM-v1-framework cost** from
+   **connector-implementation cost** by including a no-op connector.
+2. **Validate (or refute) the agentic-coupling amplification** claim:
+   if single-instance synthetic numbers ≈ 8-instance trace numbers
+   (38–45%), the coupling is not the main cause. If single-instance is
+   much smaller, then the 8-instance saturated coupling does most of
+   the damage.
+3. Make the result **reproducible and auditable**: every run dumps full
+   raw artifacts + manifest entry + a re-run script.
+
+---
+
+## Hypotheses (revised based on elastic_migration_v2 prior)
+
+The headline trace-replay numbers from elastic_migration_v2 are our
+**prior**, not an open question:
+
+```
+trace replay, 8 instances, agentic dispatch coupling, saturated:
+  plain TTFT p90 = 7.35 s
+  NIXL  TTFT p90 = ~10.1 s   (+38%)
+  Mooncake_both  = 10.67 s   (+45%)
+```
+
+The microbench validates / refutes / refines these:
+
+| ID | Hypothesis | Falsifier |
+|---|---|---|
+| **H1: Substrate tax persists at single instance / synthetic load** | Single-instance Mooncake_both TTFT p90 is ≥ 10% higher than plain at the reference rate | If <10% → trace-replay tax is dominated by 8-instance feedback coupling, not connector machinery |
+| **H2: NIXL-vs-Mooncake gap is mechanism-side, not coupling-side** | Single-instance numbers preserve the ~7 pp gap (NIXL tax < Mooncake tax by 5–10 pp) | If gap shrinks/inverts → the gap was a coupling artifact |
+| **H3: Framework-vs-implementation split** | `noop_connector` (v1 framework only, all hooks return no-op) tax is < 50% of Mooncake_both tax | Lets us attribute cost between vLLM's connector dispatch loop and the specific connector's per-step work |
+| **H4: MultiConnector tax is additive** | tax(Mooncake+LMCache) ≈ tax(Mooncake) + tax(LMCache), within 30% | If super-additive → cross-connector interference; if sub-additive → some shared per-step cost is amortized |
+| **H5: Tax is shape-dependent** | tax_TTFT_p90 grows monotonically with input length for Mooncake_both | Confirms E2 audit §6.5 hypothesis (`set(cache.keys())` walks scale with cache size) |
+| **H6: Tax compounds in decode** | tax_TPOT_p90 grows with output length | Confirms connector code runs each decode step |
+
+H3 and H4 are the must-have new hypotheses that pulled in the new configs.
+
+---
+
+## Hardware & Model
+
+| Parameter | Value |
+|---|---|
+| GPU | NVIDIA H20 96 GB × 1 (single instance) |
+| Model | Qwen3-Coder-30B-A3B-Instruct |
+| TP | 1 |
+| `max_model_len` | 200 000 |
+| `enable_prefix_caching` | true |
+| `enable_chunked_prefill` | true |
+| `max_num_batched_tokens` | 8192 |
+| `gpu_memory_utilization` | 0.9 |
+
+Single GPU per run. Each configuration is a fresh vLLM launch on GPU 0.
+
+---
+
+## Configurations (8 total, was 6)
+
+| ID | Connector | Role | Why we measure it |
+|---|---|---|---|
+| `plain` | (none) | — | Baseline |
+| `noop_connector` | custom `NoOpConnector` (this microbench ships it) | n/a | Isolate **vLLM-v1 framework** cost (build_connector_meta, mixin dispatch, get_finished bookkeeping) without any real connector work — see Note 1 |
+| `mooncake_producer` | MooncakeConnector | `kv_producer` | Isolate P-side stack |
+| `mooncake_consumer` | MooncakeConnector | `kv_consumer` | Isolate D-side stack — pre-flight gated, see §Pre-flight |
+| `mooncake_both` | MooncakeConnector | `kv_both` | The README claim |
+| `nixl_both` | NIXLConnector | `kv_both` | Connector-specific vs framework cost |
+| `lmcache_only` | `LMCacheConnectorV1` | n/a | NEW — gives H4 a denominator |
+| `multi_mooncake_lmcache` | MultiConnector(Mooncake `kv_both` + `LMCacheConnectorV1`) | mixed | Stacked-connector check (gated by pre-flight) |
+
+**Note 1 — noop_connector (we ship it, not the vLLM-bundled one)**:
+The vLLM-shipped `ExampleConnector` is NOT a true no-op — it
+implements a debug-grade disk KV cache: stores match metadata,
+serializes safetensors per-layer in `save_kv_layer`, etc. (see
+`third_party/vllm/.../example_connector.py:345`,
+`example_connector.py:250`,
+`kv_transfer_utils.py:49`). Using it would conflate framework cost
+with disk-I/O + per-layer save cost.
+
+Instead we ship `microbench/connector_tax/tools/noop_connector.py`
+that subclasses `KVConnectorBase_V1` and returns no-op for **every**
+hook:
+
+```python
+class NoOpConnector(KVConnectorBase_V1):
+    def get_num_new_matched_tokens(self, req, num_computed): return 0, False
+    def update_state_after_alloc(self, *_args, **_kw): pass
+    def build_connector_meta(self, scheduler_output): return KVConnectorMetadata()
+    def request_finished(self, *_args, **_kw): return False, None
+    def start_load_kv(self, *_args, **_kw): pass
+    def wait_for_layer_load(self, *_args, **_kw): pass
+    def save_kv_layer(self, *_args, **_kw): pass
+    def wait_for_save(self): pass
+    def get_finished(self, *_args, **_kw): return None, None
+```
+
+vLLM loads it via:
+
+```
+--kv-transfer-config '{
+    "kv_connector_module_path":
+        "microbench.connector_tax.tools.noop_connector:NoOpConnector",
+    "kv_role": "kv_both"
+}'
+```
+
+`PYTHONPATH` is set in `launch_noop_connector.sh` so vLLM can resolve
+the dotted import path.
+
+If `noop_connector` overhead ≈ 0 → all substrate tax is in connector
+implementations. If `noop_connector` overhead ≈ 30% of Mooncake_both
+tax → vLLM's framework dispatch alone explains a meaningful slice.
+
+---
+
+## Pre-flight Verification (NEW — gates risky configs)
+
+Two configs depend on infrastructure we can't take for granted. Run
+verification scripts BEFORE the main bench. Skip the config (and record
+SKIP in manifest) if it fails.
+
+### `verify_kv_consumer.sh`
+
+1. Start a dummy bootstrap process (`tools/dummy_bootstrap.py`).
+2. Launch vLLM with `kv_role=kv_consumer` pointing at the dummy.
+3. Curl `/v1/models` — must return 200 with the model id.
+4. Send one short request (`max_tokens=4`) without `kv_transfer_params`
+   — must return 200 in <30 s.
+
+If steps 3 or 4 fail, the config is unrunnable and we drop it. We do
+not try harder; the trace-replay paper does not promise consumer-only
+single-instance numbers.
+
+### `verify_multi_connector.sh`
+
+1. Launch vLLM with `MultiConnector(MooncakeConnector kv_both,
+   LMCacheConnectorV1)`.
+2. Send 5 sequential requests, `max_tokens=32`, random content.
+3. All 5 must complete in <60 s.
+4. Verify no engine crashes: `vllm:engine_core_failed_total == 0` from
+   `/metrics`.
+
+If any check fails, drop the config and mark SKIP (Manifest column:
+"why skipped").
+
+### `verify_noop_connector.sh`
+
+1. Launch with noop_connector active (loaded via `kv_connector_module_path`).
+2. Send 5 sequential requests, `max_tokens=32`.
+3. Verify all 5 return 200 in <30 s and no engine crash.
+
+This one is unlikely to fail but the verification is the same.
+
+The verification scripts produce `verify_<config>.log` under
+`results/preflight/` and a `preflight_status.json` summarizing
+skip-or-include decisions for the manifest.
+
+---
+
+## Workload (revised)
+
+**Open-loop, fixed-rate, randomized content, two-phase sweep,
+data-driven saturation criteria.**
+
+### Phase A — rate sweep (find saturation per config)
+
+| Parameter | Value |
+|---|---|
+| Input length | 4096 tokens (random per request) |
+| Output length | 256 tokens (`max_tokens=256`, `ignore_eos=True`) |
+| Send rates | {0.5, 1, 2, 4, 8, 16, 32} req/s (added 0.5 for low-end calibration) |
+| Duration per cell | `max(60 s, time_to_min_completed)` + 10 s warmup |
+| Min completed per cell | 200 requests |
+| Inflight cap | 256 (drop excess to log) |
+
+**Why min_completed = 200**: at p90, the margin-of-error of a Monte
+Carlo percentile estimate from N samples is ≈ 1.65 √(0.9 × 0.1 / N).
+For N=200 this is ~3.5% absolute, ~10% relative — acceptable. For
+N=30 (which 0.5 req/s × 60 s gives) it's ~28% relative, useless for
+saturation detection. So at low rates the cell automatically extends:
+0.5 req/s → ≥ 400 s, 1 req/s → ≥ 200 s. At 4 req/s and above the
+60-second floor dominates.
+
+Updated Phase A duration per config (rounded):
+
+| Rate (req/s) | Duration (s) | Note |
+|---|---|---|
+| 0.5 | 410 | extended to hit 200 completed |
+| 1   | 210 | extended |
+| 2   | 110 | extended slightly |
+| 4   | 70  | floor |
+| 8   | 70  | floor |
+| 16  | 70  | floor |
+| 32  | 70  | floor |
+| **sum per config (excl. warmup)** | **~1010 s** | |
+
+Total Phase A: 8 configs × (90 s vLLM warmup + 1010 s of cells +
+60 s GPU release) = 8 × 1160 s ≈ **155 min**.
+
+### Saturation criteria (data-driven, was hardcoded inflight>8)
+
+A config is **saturated at rate r** if **any** of:
+
+1. `effective_throughput(r) / r < 0.95` — vLLM can't keep up
+2. `num_requests_waiting p50 (from /metrics) > 1` — vLLM has visible queue
+3. `TTFT p90 (r) / TTFT p90 (r/2) > 1.5` — TTFT inflating super-linearly
+
+The **per-config saturation rate** is the lowest r that triggers ≥ 1
+criterion. We log which criterion fired so reviewers can disagree.
+
+### Reference rate selection (revised)
+
+We define **two reference rates** for Phase B, both computed from
+Phase A data:
+
+```
+ref_safe = max rate where ALL 8 configs are NOT saturated
+ref_load = max rate where plain is NOT saturated
+           (some other configs may be saturated here)
+```
+
+`ref_safe` measures the **pure substrate per-step tax** under no
+queueing.
+
+`ref_load` measures **the tax in the regime closer to deployment** —
+where plain is happily under-loaded but Mooncake is starting to hurt.
+The gap `tax(ref_load) − tax(ref_safe)` is the **non-linear queueing
+amplification** of the substrate tax. This is exactly the effect the
+reviewer worried about and now we measure it explicitly instead of
+ignoring it.
+
+Both rates are reported. The headline number we cite is `ref_safe`
+because it's the cleanest decomposition. The `ref_load` numbers tell
+us how much worse the tax gets near saturation.
+
+### Phase B — shape sweep (substrate tax across length regimes)
+
+| Parameter | Value |
+|---|---|
+| Send rate | `ref_safe` (one value, single rate to keep cost bounded) |
+| Input lengths | {512, 4096, 32768} tokens |
+| Output lengths | {64, 256, 1024} tokens (32 promoted to 64 — see Note 2) |
+| Duration per cell | `max(60 s, time_to_min_completed)` + 10 s warmup |
+| Min completed per cell | 200 requests |
+| Cartesian shapes | 3 × 3 = 9 |
+
+The same min-completed extension applies. If `ref_safe ≥ 4 req/s`,
+each cell hits the 60 s floor and per-config Phase B cell time is
+9 × 70 s = 630 s. If `ref_safe = 2 req/s`, cells extend to 110 s and
+per-config cell time is 9 × 110 s = 990 s.
+
+Total Phase B (worst case, `ref_safe = 2`): 8 configs × (90 s warmup
++ 990 s of cells + 60 s GPU release) ≈ **152 min**. Best case
+(`ref_safe ≥ 4`): 8 × (90 + 630 + 60) ≈ **104 min**.
+
+If after Phase A we find `ref_load` differs meaningfully from
+`ref_safe`, we add a small Phase B' run on `ref_load` for the 4
+high-priority configs (plain, mooncake_both, nixl_both, lmcache_only)
+on 3 representative shapes (512/256, 4096/256, 32768/256). That is
+4 configs × 3 shapes × 70 s ≈ 14 min, controlled trade-off.
+
+**Note 2 — output 64 instead of 32**: with 32 output tokens TPOT is
+estimated from 31 inter-token intervals — too few samples for stable
+p90. Bumping to 64 gives 63 samples, comfortable for percentile
+estimation. The output=32 regime is also less common in agentic
+deployments where a tool result frame is rarely <64 tokens.
+
+### Common settings
+
+| Parameter | Value |
+|---|---|
+| `temperature` | 0 (deterministic) |
+| `ignore_eos` | True (force exact output length) |
+| Content | random UUID + hash per request, zero prefix cache hit |
+| Concurrent inflight cap | 256 |
+
+---
+
+## Metrics (revised — adds A3 step-level engine_state)
+
+### Client-side (per-request, JSONL)
+
+Same as before: `t_send_ns`, `t_first_token_ns`, `t_last_token_ns`,
+`prompt_tokens`, `completion_tokens`, `inflight_at_send`.
+
+### Server-side `/metrics` sampling (1 Hz)
+
+Captured into `metrics_<cfg>_<phase>_<cell>.jsonl`. Same fields as
+prior version.
+
+### Step-level timing instrumentation (NEW — we ship the patch)
+
+The reviewer correctly noted that the existing A3 step log
+(`third_party/vllm/.../scheduler.py:953`) only records per-step token
+counts and request lists, **not** step duration or per-callback
+timing. So we cannot just turn on AGENTIC_STEP_LOG_PATH and get
+Figure 6/7's "direct evidence" — that data does not exist yet.
+
+This microbench ships its own scheduler timing patch at
+`microbench/connector_tax/patches/scheduler_step_timing.py`, modelled
+on the idempotent `microbench/patches/apply_patches.py` we wrote for
+Microbench 2. It uses the same `_pd_profile.py` emit pattern.
+
+The patch instruments:
+
+1. `Scheduler.schedule()` entry → `t_step_enter` (perf_counter_ns)
+2. `Scheduler.schedule()` exit → `t_step_exit`
+3. Around `connector.build_connector_meta(scheduler_output)`
+   → `build_meta_us`
+4. Around `connector.get_finished(...)` call
+   (in `_update_from_output` / mixin)
+   → `get_finished_us`
+5. Around `connector.start_load_kv(...)` (in the worker mixin
+   `_get_kv_connector_output`)
+   → `start_load_kv_us` (worker-side; emitted from worker process)
+
+Each step emits one JSONL record:
+
+```json
+{
+  "t_ns": <step_enter perf_counter_ns>,
+  "step_id": <monotonic int>,
+  "step_duration_us": <step_exit - step_enter>,
+  "build_meta_us":    <build_connector_meta duration>,
+  "get_finished_us":  <connector get_finished duration>,
+  "start_load_kv_us": <worker start_load_kv; null on scheduler-only proc>,
+  "num_running":      <int>,
+  "num_waiting":      <int>,
+  "prefill_tokens":   <int>,
+  "decode_tokens":    <int>
+}
+```
+
+Output goes to `AGENTIC_STEP_LOG_PATH` (one file per process; we use
+`engine_step_<phase>_<cell>.jsonl` paths from launch scripts).
+
+Apply / revert is idempotent — same `# CONNECTOR_TAX_PATCH` marker
+strategy as Microbench 2.
+
+```
+microbench/connector_tax/patches/
+├── _step_profile.py         # the emitter (ported from _pd_profile)
+├── scheduler_step_timing.py # patch installer / reverter
+└── apply.sh                 # invoked by run_all.sh; revert at end
+```
+
+**Fallback if the patch fails to apply on a future vLLM version**:
+the bench drops to client-side TTFT/TPOT only. Figures 6 (per-step
+CDF) and 7 (decomposition stack) are not produced; the manifest
+records `step_timing_available=false`. The other figures and the
+H1 / H2 / H4 headline numbers do not depend on this patch, so the
+bench is still useful in fallback mode.
+
+### Derived (post-processing)
+
+For each (config, rate-or-shape) cell after warmup:
+
+- TTFT/TPOT/E2E p50/p90/p99
+- `effective_throughput`, `requested_throughput`, throughput_ratio
+- `saturation_flag` (which criterion, if any, triggered)
+- (when `step_timing_available=true`):
+  - `step_duration_us` p50/p90
+  - `build_meta_us` p50/p90
+  - `get_finished_us` p50/p90
+  - `start_load_kv_us` p50/p90 (worker-process file)
+  - `connector_total_us` p50/p90  (sum of the 3 callback timings)
+
+### Substrate tax definition
+
+```
+tax_TTFT_p90(X, ref) = TTFT_p90(X, ref) / TTFT_p90(plain, ref) - 1
+tax_TPOT_p90(X, ref) = TPOT_p90(X, ref) / TPOT_p90(plain, ref) - 1
+tax_step_p50(X)      = step_duration_us p50 (X) - step_duration_us p50 (plain)
+tax_callback_p50(X)  = connector_total_us p50 (X)   # plain has no callbacks
+```
+
+`tax_step` is the **gross** per-step penalty (any cause).
+`tax_callback` is the **callback-attributable** penalty (sum of the
+three measured connector hooks). The difference `tax_step −
+tax_callback` is "step-time overhead not attributable to instrumented
+callbacks" — block-pool walks, scheduler-state churn, etc. Reporting
+both lets reviewers see whether our instrumentation accounts for the
+full cost.
+
+---
+
+## Auditability & Reproducibility Plan
+
+### Run artifacts (per config × phase × cell)
+
+```
+microbench/connector_tax/results/
+  <date>_<config>/
+    config.json                  # parameters used
+    launch.sh                    # exact vLLM launch command
+    vllm_stdout.log              # full vLLM stdout
+    vllm_stderr.log              # full vLLM stderr
+    requests_<phase>_<cell>.jsonl
+    metrics_<phase>_<cell>.jsonl
+    engine_step_<phase>_<cell>.jsonl  # if A3 active
+    summary.json                 # per-cell percentiles
+    env.txt                      # pip freeze, vLLM SHA, GPU info
+  preflight/
+    verify_kv_consumer.log
+    verify_multi_connector.log
+    verify_noop_connector.log
+    preflight_status.json        # which configs are SKIP'd and why
+```
+
+### Manifest
+
+`microbench/connector_tax/MANIFEST.md` lists every run with date,
+vLLM version + git SHA, Mooncake version, NIXL version, LMCache
+version, GPU id (`nvidia-smi -L`), config name, launch command, result
+directory, A3-active flag, and skip-status (with reason).
+
+### Re-run script
+
+`microbench/connector_tax/run_all.sh` runs in **three barrier stages**.
+Phase A across all configs must finish before Phase B can pick a
+reference rate.
+
+**Stage 0 — Pre-flight + patch:**
+1. Run `verify_kv_consumer.sh`, `verify_multi_connector.sh`, and
+   `verify_noop_connector.sh`. Persist `preflight_status.json`.
+2. Apply `microbench/connector_tax/patches/scheduler_step_timing.py`
+   to the active vLLM. Record `step_timing_available=true|false`
+   in the manifest based on whether the patch applied cleanly.
+
+**Stage 1 — Phase A (all configs, randomized order):**
+For each non-SKIP config:
+1. `launch_<config>.sh` → wait for `/v1/models`.
+2. `bench_loop.py --rates 0.5,1,2,4,8,16,32 --shape 4096,256
+   --duration 60 --min-completed 200`.
+3. Kill vLLM, wait 60 s for GPU release.
+4. Append manifest row.
+
+After **all** configs have finished Stage 1:
+
+**Stage 2 — Reference rate selection (CPU only):**
+1. Compute saturation flags from each cell using the data-driven
+   criteria.
+2. Choose `ref_safe` = max rate where ALL configs that completed
+   Phase A are not saturated.
+3. Choose `ref_load` = max rate where `plain` is not saturated.
+4. Persist `reference_rates.json`.
+
+**Stage 3 — Phase B (all configs, randomized order):**
+For each non-SKIP config:
+1. `launch_<config>.sh` → wait for ready.
+2. `bench_loop.py --rate <ref_safe> --shapes 512x64,512x256,
+   ...,32768x1024 --duration 60 --min-completed 200`.
+3. (If `ref_load != ref_safe`) Run Phase B' for priority configs
+   (plain, mooncake_both, nixl_both, lmcache_only) on shapes
+   {512x256, 4096x256, 32768x256} at `ref_load`.
+4. Kill vLLM, wait 60 s, append manifest row.
+
+**Stage 4 — Patch revert + analysis:**
+1. Revert the scheduler_step_timing patch.
+2. `analyze.py --root results/`.
+3. `plot_connector_tax.py`.
+
+A reviewer with a fresh checkout runs:
+
+```
+cd microbench/connector_tax
+bash run_all.sh
+```
+
+and gets the figures + manifest + raw artifacts. The script is
+re-runnable: any stage can be skipped via `--skip-stage N` if the
+artifacts exist.
+
+### Determinism notes
+
+Same as previous: temperature=0 + ignore_eos give shape determinism;
+content varies per request via seeded UUID. We do not promise
+bit-exact reproducibility, only distribution-level reproducibility.
+
+### Updated runtime estimate (was 1.5–2 h, **now 4–5.5 h**)
+
+| Phase | Time |
+|---|---|
+| Pre-flight (3 verify scripts) | 15 min |
+| Phase A: 8 configs × (90 s warmup + 1010 s cells + 60 s GPU clear) | 155 min |
+| Phase A → ref_safe selection (CPU) | <1 min |
+| Phase B (best, `ref_safe ≥ 4`): 8 × (90 + 630 + 60) | 104 min |
+| Phase B (worst, `ref_safe = 2`): 8 × (90 + 990 + 60) | 152 min |
+| Optional Phase B' (4 configs × 3 shapes × ≥70 s + 4 × 90 s warmup) | 20 min |
+| Analysis + figures | 5 min |
+| **Total (best case)** | **~5 h** |
+| **Total (worst case)** | **~5.5 h** |
+
+This is honest. The reviewer's earlier estimate of 2.5–3 h
+underestimated how long low-rate cells must run to give stable p90.
+
+---
+
+## Analysis & Figures
+
+### Figure 1: TTFT p90 vs send rate, per configuration (Phase A)
+
+Same as before, now 8 lines plus saturation markers (× per criterion).
+
+### Figure 2: TPOT p90 vs send rate, per configuration (Phase A)
+
+Same.
+
+### Figure 3: Achieved throughput vs requested (Phase A)
+
+Same, 8 lines + y=x reference + saturation knee annotations.
+
+### Figure 4: Substrate tax bar (TTFT p90 + TPOT p90)
+
+At `ref_safe` and `ref_load`, side-by-side bars per non-plain config.
+Shows:
+- Pure tax (`ref_safe`)
+- Tax + non-linear queueing (`ref_load`)
+- The gap is the **coupling amplification** the reviewer flagged.
+
+### Figure 5: Shape-dependent tax heatmap (Phase B)
+
+3×3 heatmap (input × output) of tax_TTFT_p90 for each non-plain
+config. 6 heatmaps in a row, including noop_connector,
+mooncake_both, nixl_both, lmcache_only, mooncake_producer,
+multi_mooncake_lmcache. (Skip mooncake_consumer if pre-flight
+dropped it.)
+
+### Figure 6: Per-step latency CDF, ref_safe rate (Phase A)
+
+X = step duration (μs), Y = CDF, line per config. **The most direct
+visualization of "what each step costs."** Shipped only if A3 step log
+is available.
+
+### Figure 7: Tax decomposition stack
+
+For each non-plain config at ref_safe, stacked bar:
+- "framework cost" estimated = tax(noop_connector)
+- "implementation cost" estimated = tax(this config) − tax(noop_connector)
+
+If `noop_connector` doesn't run (we'd document why), we drop this
+figure and report tax as a single number per config.
+
+### Figure 8: H4 additivity check
+
+3-bar group: tax(mooncake_both), tax(lmcache_only), tax(multi). The
+sum-of-first-two compared against multi visualizes additivity.
+
+---
+
+## Risks & Mitigations (revised)
+
+| Risk | Impact | Mitigation |
+|---|---|---|
+| `kv_consumer` won't start with dummy bootstrap | Skip the config | Pre-flight; documented SKIP in manifest |
+| `multi_mooncake_lmcache` crashes engine | Skip the config | Pre-flight |
+| NIXL not installed | Skip nixl_both | Tolerant; warn + continue |
+| LMCache not installed | Skip lmcache_only AND multi config | Tolerant; warn + continue |
+| GPU thermal drift across 3+ h | Skews late configs | Run order randomized; consider running twice on different days and reporting both |
+| Open-loop blow-up at 32 req/s | Memory blowup | Inflight cap 256, drop with logged counter |
+| Cold-start of first request | Inflates mean TTFT | 10 s warmup discarded |
+| `scheduler_step_timing` patch fails to apply on a future vLLM version | Lose Figures 6 and 7 | Document `step_timing_available=false` in manifest; H1/H2/H4 still report from client-side TTFT/TPOT |
+| `noop_connector` import fails (PYTHONPATH or class signature) | Lose Figures 7 + H3 falsifier | Pre-flight `verify_noop_connector.sh` catches this; report SKIP in manifest |
+
+---
+
+## Success Criteria (revised)
+
+1. **H1 falsifiable**: tax_TTFT_p90 for `mooncake_both` at `ref_safe`
+   is reported. We accept the prior (≈45%) if measurement is in
+   [25%, 60%]; we **revise the prior** if outside.
+2. **H2 testable**: NIXL-vs-Mooncake gap at `ref_safe` is reported.
+   The trace-replay difference was ~7 pp. We document agreement /
+   disagreement.
+3. **H3 disambiguated**: tax(noop_connector) at `ref_safe` is
+   reported. We label substrate tax as
+   "framework-cost-dominated" if noop_connector ≥ 50% of
+   mooncake_both tax, "implementation-cost-dominated" if < 30%.
+4. **H4 additivity**: |tax(multi) − (tax(mooncake_both) +
+   tax(lmcache_only))| / tax(multi) ≤ 0.30 → linear.
+5. **H5 + H6 directional**: report whether tax_TTFT_p90 grows with
+   input and tax_TPOT_p90 grows with output (sign + magnitude).
+6. **All artifacts present**: every config that ran has the 6 file
+   types; every SKIP config has a reason in `preflight_status.json`.
+7. **Bench finishes < 6 h** wall clock on idle dash0
+   (Phase A + Phase B + optional Phase B' combined; reflects min-completed
+   extension at low rates).
+
+---
+
+## Out of Scope
+
+- Multi-node Mooncake (RDMA over actual network).
+- Patching Mooncake or vLLM to optimize the substrate (the point of
+  this microbench is to measure baseline cost as shipped).
+- Varying `chunk_size`, `max_num_seqs`, or other vLLM scheduler
+  parameters; fixed at trace-replay defaults.
+- chunk-boundary effects (input ∈ {8192, 16384}). The reviewer noted
+  this is a real follow-up but adding it doubles Phase B runtime.
+  Documented as a follow-up if Phase B shows shape-dependent tax that
+  can't be explained by total token count.
+
+---
+
+## Cross-references
+
+- `analysis/characterization/elastic_migration_v2/README.md` — the
+  trace-replay paper this microbench validates / refutes.
+- `microbench/interference/` — Microbench 1 (B2 same-worker
+  interference; complementary).
+- `microbench/lifecycle/` — Microbench 2 (PD-sep transfer breakdown;
+  uses different vLLM patches).
+- `microbench/patches/` — `_pd_profile.py` template if A3 fallback
+  is needed.
+
+---
+
+## Files
+
+```
+microbench/connector_tax/
+├── DESIGN.md                       # this file
+├── MANIFEST.md                     # filled per run
+├── tools/
+│   ├── noop_connector.py           # custom NoOpConnector for H3
+│   ├── dummy_bootstrap.py          # for kv_consumer pre-flight
+│   ├── verify_kv_consumer.sh
+│   ├── verify_multi_connector.sh
+│   └── verify_noop_connector.sh
+├── patches/
+│   ├── _step_profile.py            # event emitter (ports _pd_profile)
+│   ├── scheduler_step_timing.py    # idempotent install/revert
+│   └── apply.sh                    # invoked by run_all.sh
+├── launch/
+│   ├── launch_plain.sh
+│   ├── launch_noop_connector.sh
+│   ├── launch_mooncake_producer.sh
+│   ├── launch_mooncake_consumer.sh
+│   ├── launch_mooncake_both.sh
+│   ├── launch_nixl_both.sh
+│   ├── launch_lmcache_only.sh
+│   └── launch_multi_mooncake_lmcache.sh
+├── bench_loop.py                   # open-loop loadgen (--min-completed)
+├── metrics_sampler.py              # /metrics scraper
+├── analyze.py                      # raw → percentiles + saturation flags
+├── plot_connector_tax.py           # all figures
+├── run_all.sh                      # 4-stage barrier orchestrator
+└── results/<date>_<config>/        # per-run artifacts
+└── results/preflight/              # pre-flight verification
+```

microbench/connector_tax/analyze.py

@@ -0,0 +1,177 @@
+#!/usr/bin/env python3
+"""Aggregate connector_tax results.
+
+Reads results/<config>/summary_A.json and summary_B.json for every config,
+applies saturation criteria, picks ref_safe / ref_load, and writes
+aggregate.json + aggregate.csv.
+
+Usage:
+  analyze.py --root microbench/connector_tax/results
+"""
+
+import argparse
+import csv
+import json
+from pathlib import Path
+
+
+SAT_THROUGHPUT_RATIO = 0.95
+SAT_QUEUE_P50 = 1.0
+SAT_TTFT_INFLATION = 1.5  # vs previous (lower) rate
+
+
+def saturated(cell: dict, prev_ttft_p90: float | None) -> tuple[bool, list[str]]:
+    reasons = []
+    tr = cell.get("throughput_ratio")
+    if tr is not None and tr < SAT_THROUGHPUT_RATIO:
+        reasons.append(f"throughput_ratio={tr:.2f}<{SAT_THROUGHPUT_RATIO}")
+    # queue p50 from inflight (proxy)
+    inf50 = cell.get("inflight_p50") or 0
+    # Note: inflight_p50 measured at send time. >= 2 means queue forming.
+    if inf50 >= 2:
+        # Throughput tracking is the primary signal; this is corroboration.
+        pass
+    ttft = cell.get("ttft_ms_p90")
+    if (
+        ttft is not None
+        and prev_ttft_p90 is not None
+        and prev_ttft_p90 > 0
+        and ttft / prev_ttft_p90 > SAT_TTFT_INFLATION
+    ):
+        reasons.append(f"ttft_p90 inflated {ttft / prev_ttft_p90:.2f}x")
+    return (len(reasons) > 0, reasons)
+
+
+def analyze(root: Path) -> dict:
+    configs: dict[str, dict] = {}
+    for cfg_dir in sorted(root.iterdir()):
+        if not cfg_dir.is_dir():
+            continue
+        if cfg_dir.name == "preflight":
+            continue
+        cfg = cfg_dir.name
+        sa = cfg_dir / "summary_A.json"
+        sb = cfg_dir / "summary_B.json"
+        cfg_data = {"phase_a": [], "phase_b": []}
+        if sa.exists():
+            cfg_data["phase_a"] = json.loads(sa.read_text())
+        if sb.exists():
+            cfg_data["phase_b"] = json.loads(sb.read_text())
+        configs[cfg] = cfg_data
+
+    # ── flag saturation per cell, per config (Phase A only) ────────
+    for cfg, data in configs.items():
+        cells = sorted(data["phase_a"], key=lambda c: c["rate_target"])
+        prev = None
+        for c in cells:
+            sat, reasons = saturated(c, prev)
+            c["saturated"] = sat
+            c["sat_reasons"] = reasons
+            prev = c.get("ttft_ms_p90")
+
+    # ── pick reference rates ───────────────────────────────────────
+    # ref_safe = max rate where ALL configs are NOT saturated
+    rates = sorted({c["rate_target"]
+                    for d in configs.values()
+                    for c in d["phase_a"]})
+    ref_safe = None
+    for r in rates:
+        all_ok = True
+        for cfg, d in configs.items():
+            cells = [c for c in d["phase_a"] if c["rate_target"] == r]
+            if not cells:
+                continue
+            if cells[0]["saturated"]:
+                all_ok = False
+                break
+        if all_ok:
+            ref_safe = r
+
+    # ref_load = max rate where 'plain' is not saturated
+    ref_load = None
+    plain = configs.get("plain", {})
+    for c in sorted(plain.get("phase_a", []), key=lambda c: c["rate_target"]):
+        if not c["saturated"]:
+            ref_load = c["rate_target"]
+
+    out = {
+        "configs": configs,
+        "rates_swept": rates,
+        "ref_safe": ref_safe,
+        "ref_load": ref_load,
+    }
+    return out
+
+
+def write_csv(agg: dict, out_path: Path) -> None:
+    rows = []
+    for cfg, d in agg["configs"].items():
+        for c in d["phase_a"]:
+            rows.append({
+                "config": cfg,
+                "phase": "A",
+                "rate": c["rate_target"],
+                "input_tokens": c["input_tokens"],
+                "output_tokens": c["output_tokens"],
+                "ttft_p50": c.get("ttft_ms_p50"),
+                "ttft_p90": c.get("ttft_ms_p90"),
+                "ttft_p99": c.get("ttft_ms_p99"),
+                "tpot_p50": c.get("tpot_ms_p50"),
+                "tpot_p90": c.get("tpot_ms_p90"),
+                "tpot_p99": c.get("tpot_ms_p99"),
+                "e2e_p90":  c.get("e2e_ms_p90"),
+                "throughput_eff": c.get("throughput_effective_rps"),
+                "throughput_ratio": c.get("throughput_ratio"),
+                "n_after_warmup": c.get("n_after_warmup"),
+                "saturated": c.get("saturated"),
+                "sat_reasons": ";".join(c.get("sat_reasons", [])),
+            })
+        for c in d["phase_b"]:
+            rows.append({
+                "config": cfg,
+                "phase": "B",
+                "rate": c["rate_target"],
+                "input_tokens": c["input_tokens"],
+                "output_tokens": c["output_tokens"],
+                "ttft_p50": c.get("ttft_ms_p50"),
+                "ttft_p90": c.get("ttft_ms_p90"),
+                "ttft_p99": c.get("ttft_ms_p99"),
+                "tpot_p50": c.get("tpot_ms_p50"),
+                "tpot_p90": c.get("tpot_ms_p90"),
+                "tpot_p99": c.get("tpot_ms_p99"),
+                "e2e_p90":  c.get("e2e_ms_p90"),
+                "throughput_eff": c.get("throughput_effective_rps"),
+                "throughput_ratio": c.get("throughput_ratio"),
+                "n_after_warmup": c.get("n_after_warmup"),
+                "saturated": "",
+                "sat_reasons": "",
+            })
+
+    if not rows:
+        return
+    fields = list(rows[0].keys())
+    with open(out_path, "w", newline="") as f:
+        w = csv.DictWriter(f, fieldnames=fields)
+        w.writeheader()
+        w.writerows(rows)
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--root", type=Path, required=True)
+    ap.add_argument("--out", type=Path, default=None)
+    args = ap.parse_args()
+
+    if not args.root.exists():
+        raise SystemExit(f"root not found: {args.root}")
+
+    agg = analyze(args.root)
+    out = args.out or args.root / "aggregate.json"
+    out.write_text(json.dumps(agg, indent=2))
+    write_csv(agg, args.root / "aggregate.csv")
+    print(f"ref_safe = {agg['ref_safe']}  ref_load = {agg['ref_load']}")
+    print(f"Wrote {out} and aggregate.csv")
+
+
+if __name__ == "__main__":
+    main()

microbench/connector_tax/bench_loop.py

@@ -0,0 +1,351 @@
+#!/usr/bin/env python3
+"""Open-loop fixed-rate loadgen for connector_tax microbench.
+
+Sends requests at a Poisson-arrival rate (rate_target req/s) of fixed
+(input_tokens, output_tokens) shape with random content. Each cell runs
+until BOTH the duration floor AND min-completed thresholds are met.
+
+Per-request metrics are appended to a JSONL file.
+
+Usage:
+  bench_loop.py \
+    --url http://127.0.0.1:8000/v1/chat/completions \
+    --model /path/to/model \
+    --rates 0.5,1,2,4,8 \
+    --shape 4096,256 \
+    --duration 60 \
+    --min-completed 200 \
+    --warmup 10 \
+    --output-dir results/<run> \
+    --phase A
+"""
+
+import argparse
+import asyncio
+import hashlib
+import json
+import random
+import statistics
+import time
+import uuid
+from dataclasses import asdict, dataclass, field
+from pathlib import Path
+
+import httpx
+
+
+@dataclass
+class ReqMetric:
+    req_id: str
+    rate_target: float
+    input_tokens_target: int
+    output_tokens_target: int
+    t_send_ns: int
+    t_first_token_ns: int | None = None
+    t_last_token_ns: int | None = None
+    prompt_tokens: int = 0
+    completion_tokens: int = 0
+    inflight_at_send: int = 0
+    error: str | None = None
+
+
+# ─── content generation ────────────────────────────────────────────────────
+def make_random_prompt(target_tokens: int) -> str:
+    """Generate a prompt that tokenizes to roughly target_tokens.
+
+    Calibration (Qwen3-Coder tokenizer): "Block N: <32-hex>" ≈ 35 tokens.
+    """
+    n_parts = max(1, target_tokens // 35)
+    seed = uuid.uuid4().hex
+    parts = []
+    for i in range(n_parts):
+        h = hashlib.md5(f"{seed}_{i}_{time.time_ns()}".encode()).hexdigest()
+        parts.append(f"Block {i}: {h}")
+    return " ".join(parts)
+
+
+# ─── single request worker ─────────────────────────────────────────────────
+async def send_one(client: httpx.AsyncClient, url: str, model: str,
+                   inp_tokens: int, out_tokens: int,
+                   rate: float, inflight: list[int],
+                   inflight_cap: int) -> ReqMetric:
+    rid = uuid.uuid4().hex[:16]
+
+    if inflight[0] >= inflight_cap:
+        # Drop with logged metric
+        return ReqMetric(
+            req_id=rid, rate_target=rate,
+            input_tokens_target=inp_tokens, output_tokens_target=out_tokens,
+            t_send_ns=time.perf_counter_ns(),
+            inflight_at_send=inflight[0],
+            error="dropped_due_to_inflight_cap",
+        )
+
+    inflight[0] += 1
+    m = ReqMetric(
+        req_id=rid, rate_target=rate,
+        input_tokens_target=inp_tokens, output_tokens_target=out_tokens,
+        t_send_ns=time.perf_counter_ns(),
+        inflight_at_send=inflight[0],
+    )
+
+    try:
+        prompt = make_random_prompt(inp_tokens)
+        payload = {
+            "model": model,
+            "messages": [{"role": "user", "content": prompt}],
+            "max_tokens": out_tokens,
+            "min_tokens": out_tokens,
+            "temperature": 0,
+            "ignore_eos": True,
+            "stream": True,
+            "stream_options": {"include_usage": True},
+        }
+        async with client.stream("POST", url, json=payload, timeout=600.0) as resp:
+            resp.raise_for_status()
+            async for line in resp.aiter_lines():
+                if not line.startswith("data: "):
+                    continue
+                data = line[6:]
+                if data.strip() == "[DONE]":
+                    break
+                try:
+                    chunk = json.loads(data)
+                except json.JSONDecodeError:
+                    continue
+                # Capture usage from any chunk (it may arrive in a trailing
+                # chunk with empty `choices`).
+                usage = chunk.get("usage")
+                if usage:
+                    m.prompt_tokens = usage.get("prompt_tokens", m.prompt_tokens)
+                    m.completion_tokens = usage.get(
+                        "completion_tokens", m.completion_tokens)
+                choices = chunk.get("choices") or []
+                if not choices:
+                    continue
+                delta = choices[0].get("delta", {})
+                if "role" in delta:
+                    continue
+                now = time.perf_counter_ns()
+                if m.t_first_token_ns is None:
+                    m.t_first_token_ns = now
+                m.t_last_token_ns = now
+    except Exception as e:
+        m.error = f"{type(e).__name__}: {e}"
+    finally:
+        inflight[0] -= 1
+
+    return m
+
+
+# ─── per-cell driver (one rate × shape) ────────────────────────────────────
+async def run_cell(
+    client: httpx.AsyncClient,
+    url: str,
+    model: str,
+    rate: float,
+    inp_tokens: int,
+    out_tokens: int,
+    duration_floor_s: float,
+    min_completed: int,
+    warmup_s: float,
+    inflight_cap: int,
+    out_path: Path,
+) -> dict:
+    """Run one (rate, shape) cell. Streams per-request JSONL to out_path.
+    Returns aggregated summary."""
+
+    inflight = [0]
+    metrics: list[ReqMetric] = []
+    pending_tasks: list[asyncio.Task] = []
+    t_start_ns = time.perf_counter_ns()
+    cell_start = time.perf_counter()
+
+    print(f"  [cell] rate={rate} shape=({inp_tokens},{out_tokens}) "
+          f"floor={duration_floor_s}s min_completed={min_completed}")
+
+    interval_mean = 1.0 / rate
+    rng = random.Random(int(time.time_ns()) & 0xFFFFFFFF)
+
+    fh = open(out_path, "a", buffering=1)
+    completed_count = 0
+
+    def reap_one(t: asyncio.Task) -> None:
+        nonlocal completed_count
+        try:
+            m = t.result()
+        except Exception:
+            return
+        metrics.append(m)
+        fh.write(json.dumps(asdict(m)) + "\n")
+        completed_count += 1
+
+    async def submit():
+        while True:
+            elapsed = time.perf_counter() - cell_start
+            if elapsed >= duration_floor_s and completed_count >= min_completed:
+                return
+            task = asyncio.create_task(
+                send_one(client, url, model, inp_tokens, out_tokens,
+                         rate, inflight, inflight_cap)
+            )
+            pending_tasks.append(task)
+            await asyncio.sleep(rng.expovariate(1.0 / interval_mean))
+
+    submitter = asyncio.create_task(submit())
+
+    async def drain_periodic():
+        while not submitter.done():
+            keep = []
+            for t in pending_tasks:
+                if t.done():
+                    reap_one(t)
+                else:
+                    keep.append(t)
+            pending_tasks[:] = keep
+            await asyncio.sleep(0.1)
+
+    drainer = asyncio.create_task(drain_periodic())
+    await submitter
+    drainer.cancel()
+    try:
+        await drainer
+    except asyncio.CancelledError:
+        pass
+
+    # Final drain: wait for all remaining inflight to complete and write them
+    if pending_tasks:
+        await asyncio.gather(*pending_tasks, return_exceptions=True)
+        for t in pending_tasks:
+            if t.done():
+                reap_one(t)
+        pending_tasks.clear()
+    fh.close()
+
+    # Discard warmup window (first warmup_s seconds of completions)
+    warmup_cutoff_ns = t_start_ns + int(warmup_s * 1e9)
+    after = [m for m in metrics if m.t_send_ns > warmup_cutoff_ns and m.error is None
+             and m.t_first_token_ns and m.t_last_token_ns]
+
+    def pct(xs, p):
+        if not xs:
+            return None
+        xs = sorted(xs)
+        k = max(0, min(len(xs) - 1, int(p / 100.0 * (len(xs) - 1))))
+        return xs[k]
+
+    ttft = [(m.t_first_token_ns - m.t_send_ns) / 1e6 for m in after]
+    tpot = []
+    for m in after:
+        if m.completion_tokens > 1 and m.t_last_token_ns and m.t_first_token_ns:
+            tpot.append((m.t_last_token_ns - m.t_first_token_ns) / 1e6
+                        / max(1, m.completion_tokens - 1))
+    e2e = [(m.t_last_token_ns - m.t_send_ns) / 1e6 for m in after]
+    inflight_seq = [m.inflight_at_send for m in after]
+    elapsed_s = (time.perf_counter_ns() - t_start_ns) / 1e9
+
+    summary = {
+        "rate_target": rate,
+        "input_tokens": inp_tokens,
+        "output_tokens": out_tokens,
+        "duration_actual_s": elapsed_s,
+        "n_completed_total": len(metrics),
+        "n_after_warmup": len(after),
+        "n_dropped": sum(1 for m in metrics if m.error == "dropped_due_to_inflight_cap"),
+        "n_errors": sum(1 for m in metrics if m.error and m.error != "dropped_due_to_inflight_cap"),
+        "ttft_ms_p50": pct(ttft, 50),
+        "ttft_ms_p90": pct(ttft, 90),
+        "ttft_ms_p99": pct(ttft, 99),
+        "tpot_ms_p50": pct(tpot, 50),
+        "tpot_ms_p90": pct(tpot, 90),
+        "tpot_ms_p99": pct(tpot, 99),
+        "e2e_ms_p50": pct(e2e, 50),
+        "e2e_ms_p90": pct(e2e, 90),
+        "e2e_ms_p99": pct(e2e, 99),
+        "throughput_effective_rps": len(after) / max(1.0, elapsed_s - warmup_s),
+        "throughput_ratio": (len(after) / max(1.0, elapsed_s - warmup_s)) / rate,
+        "inflight_p50": pct(inflight_seq, 50),
+        "inflight_p90": pct(inflight_seq, 90),
+    }
+    print(f"    completed={len(after)} ttft_p90={summary['ttft_ms_p90']} "
+          f"tpot_p90={summary['tpot_ms_p90']} thr_ratio={summary['throughput_ratio']:.2f}")
+    return summary
+
+
+async def main_async(args):
+    out_dir = Path(args.output_dir)
+    out_dir.mkdir(parents=True, exist_ok=True)
+
+    rates = [float(x) for x in args.rates.split(",")] if args.rates else [args.rate]
+    shapes = []
+    if args.shape:
+        ip, op = args.shape.split(",")
+        shapes = [(int(ip), int(op))]
+    elif args.shapes:
+        for s in args.shapes.split(","):
+            ip, op = s.split("x")
+            shapes.append((int(ip), int(op)))
+
+    summaries = []
+    timeout = httpx.Timeout(600.0)
+    async with httpx.AsyncClient(timeout=timeout) as client:
+        for rate in rates:
+            for inp, out in shapes:
+                cell_label = f"{args.phase}_r{rate}_{inp}x{out}"
+                req_path = out_dir / f"requests_{cell_label}.jsonl"
+
+                # min_completed-driven duration floor
+                min_floor_for_rate = max(1, int(args.min_completed / rate)) if rate > 0 else args.duration
+                floor = max(args.duration, min_floor_for_rate)
+
+                summary = await run_cell(
+                    client, args.url, args.model, rate, inp, out,
+                    duration_floor_s=floor,
+                    min_completed=args.min_completed,
+                    warmup_s=args.warmup,
+                    inflight_cap=args.inflight_cap,
+                    out_path=req_path,
+                )
+                summary["phase"] = args.phase
+                summary["cell"] = cell_label
+                summaries.append(summary)
+
+                # Cooldown between cells (let queue drain)
+                await asyncio.sleep(3.0)
+
+    # Persist summary
+    with open(out_dir / f"summary_{args.phase}.json", "w") as f:
+        json.dump(summaries, f, indent=2)
+    print(f"\nWrote {len(summaries)} cell summaries.")
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--url", required=True,
+                    help="vLLM /v1/chat/completions URL")
+    ap.add_argument("--model", required=True)
+    ap.add_argument("--phase", default="A")
+    # Rate spec — either --rates a,b,c (Phase A) or --rate r (Phase B)
+    ap.add_argument("--rates", default="")
+    ap.add_argument("--rate", type=float, default=4.0)
+    # Shape spec — either --shape ip,op (Phase A) or --shapes IPxOP,IPxOP,... (Phase B)
+    ap.add_argument("--shape", default="")
+    ap.add_argument("--shapes", default="")
+    ap.add_argument("--duration", type=float, default=60.0,
+                    help="Cell duration floor (seconds)")
+    ap.add_argument("--min-completed", type=int, default=200)
+    ap.add_argument("--warmup", type=float, default=10.0)
+    ap.add_argument("--inflight-cap", type=int, default=256)
+    ap.add_argument("--output-dir", required=True)
+    args = ap.parse_args()
+
+    if not (args.rates or args.rate):
+        ap.error("Provide --rates or --rate")
+    if not (args.shape or args.shapes):
+        ap.error("Provide --shape or --shapes")
+
+    asyncio.run(main_async(args))
+
+
+if __name__ == "__main__":
+    main()

microbench/connector_tax/launch/common.sh

@@ -0,0 +1,64 @@
+#!/bin/bash
+# Common environment for all connector_tax launch scripts.
+#
+# Usage: source common.sh
+# Sets: MODEL_PATH, PYTHON, PORT, LOG_DIR, COMMON_VLLM_ARGS
+
+set -euo pipefail
+
+# Resolve project root (microbench/connector_tax/launch/common.sh → ../../..)
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+CT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"          # microbench/connector_tax
+MB_DIR="$(cd "$CT_DIR/.."   && pwd)"            # microbench
+PROJ_DIR="$(cd "$MB_DIR/.." && pwd)"            # repo root
+
+VENV="$PROJ_DIR/.venv/bin"
+PYTHON="${VENV}/python"
+[[ -x "$PYTHON" ]] || PYTHON="$(command -v python3)"
+
+MODEL_PATH="${MODEL_PATH:-$HOME/models/Qwen/Qwen3-Coder-30B-A3B-Instruct}"
+PORT="${PORT:-8000}"
+GPU_ID="${GPU_ID:-0}"
+
+# Per-run log dir (caller may override RUN_DIR)
+RUN_DIR="${RUN_DIR:-$CT_DIR/results/_default}"
+mkdir -p "$RUN_DIR"
+LOG_DIR="$RUN_DIR"
+
+# PYTHONPATH so `microbench.connector_tax.tools.noop_connector:NoOpConnector`
+# resolves when launched from anywhere.
+export PYTHONPATH="${PROJ_DIR}:${PYTHONPATH:-}"
+
+# Step-timing log path (vLLM's existing scheduler emits when this is set;
+# our patch enriches the record with timing fields)
+export AGENTIC_STEP_LOG_PATH="${AGENTIC_STEP_LOG_PATH:-$LOG_DIR/engine_step.jsonl}"
+
+# Common vLLM flags shared by all configs
+COMMON_VLLM_ARGS=(
+    --model "$MODEL_PATH"
+    --host 0.0.0.0
+    --port "$PORT"
+    --tensor-parallel-size 1
+    --trust-remote-code
+    --enable-prefix-caching
+    --dtype auto
+    --gpu-memory-utilization 0.9
+    --max-model-len 200000
+    --no-enable-log-requests
+)
+
+# Wait until /v1/models returns 200, then return.
+# Times out at $1 seconds (default 240).
+wait_for_ready() {
+    local timeout="${1:-240}"
+    local i
+    for i in $(seq 1 "$timeout"); do
+        if curl -sf "http://127.0.0.1:$PORT/v1/models" >/dev/null 2>&1; then
+            echo "Server ready after ${i}s"
+            return 0
+        fi
+        sleep 1
+    done
+    echo "ERROR: Server did not become ready within ${timeout}s" >&2
+    return 1
+}

microbench/connector_tax/launch/launch_lmcache_only.sh

@@ -0,0 +1,21 @@
+#!/bin/bash
+# vLLM + LMCacheConnectorV1 alone. Skip if LMCache not installed.
+set -euo pipefail
+source "$(dirname "${BASH_SOURCE[0]}")/common.sh"
+
+if ! "$PYTHON" -c 'import lmcache' 2>/dev/null; then
+    echo "SKIP: lmcache module not importable" >&2
+    exit 42
+fi
+
+KV_CFG='{"kv_connector":"LMCacheConnectorV1","kv_role":"kv_both"}'
+
+CUDA_VISIBLE_DEVICES="$GPU_ID" \
+"$PYTHON" -m vllm.entrypoints.openai.api_server \
+    "${COMMON_VLLM_ARGS[@]}" \
+    --kv-transfer-config "$KV_CFG" \
+    > "$LOG_DIR/vllm_stdout.log" 2> "$LOG_DIR/vllm_stderr.log" &
+VLLM_PID=$!
+echo "$VLLM_PID" > "$LOG_DIR/.vllm.pid"
+echo "vLLM PID=$VLLM_PID"
+wait_for_ready 240

microbench/connector_tax/launch/launch_mooncake_both.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+# vLLM + Mooncake kv_both (alone, never transfers — README claim under test).
+set -euo pipefail
+source "$(dirname "${BASH_SOURCE[0]}")/common.sh"
+
+BOOTSTRAP_PORT="${BOOTSTRAP_PORT:-8998}"
+KV_CFG='{"kv_connector":"MooncakeConnector","kv_role":"kv_both"}'
+
+VLLM_MOONCAKE_BOOTSTRAP_PORT="$BOOTSTRAP_PORT" \
+CUDA_VISIBLE_DEVICES="$GPU_ID" \
+"$PYTHON" -m vllm.entrypoints.openai.api_server \
+    "${COMMON_VLLM_ARGS[@]}" \
+    --kv-transfer-config "$KV_CFG" \
+    > "$LOG_DIR/vllm_stdout.log" 2> "$LOG_DIR/vllm_stderr.log" &
+VLLM_PID=$!
+echo "$VLLM_PID" > "$LOG_DIR/.vllm.pid"
+echo "vLLM PID=$VLLM_PID"
+wait_for_ready 240

microbench/connector_tax/launch/launch_mooncake_consumer.sh

@@ -0,0 +1,28 @@
+#!/bin/bash
+# vLLM + Mooncake kv_consumer.
+# Pre-flight gated: if dummy bootstrap doesn't satisfy vLLM startup, this
+# config is marked SKIP and run_all.sh skips it.
+set -euo pipefail
+source "$(dirname "${BASH_SOURCE[0]}")/common.sh"
+
+DUMMY_BOOTSTRAP_PORT="${DUMMY_BOOTSTRAP_PORT:-8997}"
+
+# Start dummy bootstrap (in background) so the consumer has someone to talk to.
+"$PYTHON" "$(dirname "${BASH_SOURCE[0]}")/../tools/dummy_bootstrap.py" \
+    --port "$DUMMY_BOOTSTRAP_PORT" \
+    > "$LOG_DIR/dummy_bootstrap.log" 2>&1 &
+DUMMY_PID=$!
+echo "$DUMMY_PID" > "$LOG_DIR/.dummy.pid"
+sleep 2
+
+KV_CFG="{\"kv_connector\":\"MooncakeConnector\",\"kv_role\":\"kv_consumer\",\"kv_connector_extra_config\":{\"prefill_addr\":\"127.0.0.1:${DUMMY_BOOTSTRAP_PORT}\"}}"
+
+CUDA_VISIBLE_DEVICES="$GPU_ID" \
+"$PYTHON" -m vllm.entrypoints.openai.api_server \
+    "${COMMON_VLLM_ARGS[@]}" \
+    --kv-transfer-config "$KV_CFG" \
+    > "$LOG_DIR/vllm_stdout.log" 2> "$LOG_DIR/vllm_stderr.log" &
+VLLM_PID=$!
+echo "$VLLM_PID" > "$LOG_DIR/.vllm.pid"
+echo "vLLM PID=$VLLM_PID"
+wait_for_ready 300

microbench/connector_tax/launch/launch_mooncake_producer.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+# vLLM + Mooncake kv_producer (no D peer, never transfers).
+set -euo pipefail
+source "$(dirname "${BASH_SOURCE[0]}")/common.sh"
+
+BOOTSTRAP_PORT="${BOOTSTRAP_PORT:-8998}"
+KV_CFG='{"kv_connector":"MooncakeConnector","kv_role":"kv_producer"}'
+
+VLLM_MOONCAKE_BOOTSTRAP_PORT="$BOOTSTRAP_PORT" \
+CUDA_VISIBLE_DEVICES="$GPU_ID" \
+"$PYTHON" -m vllm.entrypoints.openai.api_server \
+    "${COMMON_VLLM_ARGS[@]}" \
+    --kv-transfer-config "$KV_CFG" \
+    > "$LOG_DIR/vllm_stdout.log" 2> "$LOG_DIR/vllm_stderr.log" &
+VLLM_PID=$!
+echo "$VLLM_PID" > "$LOG_DIR/.vllm.pid"
+echo "vLLM PID=$VLLM_PID"
+wait_for_ready 240

microbench/connector_tax/launch/launch_multi_mooncake_lmcache.sh

@@ -0,0 +1,41 @@
+#!/bin/bash
+# vLLM + MultiConnector(MooncakeConnector kv_both, LMCacheConnectorV1).
+# Skip if either is missing.
+set -euo pipefail
+source "$(dirname "${BASH_SOURCE[0]}")/common.sh"
+
+if ! "$PYTHON" -c 'import lmcache' 2>/dev/null; then
+    echo "SKIP: lmcache module not importable" >&2
+    exit 42
+fi
+
+BOOTSTRAP_PORT="${BOOTSTRAP_PORT:-8998}"
+
+# MultiConnector wraps a list of sub-connector configs.
+# Reference: vllm/distributed/kv_transfer/kv_connector/factory.py +
+# docs/features/disagg_prefill.md
+KV_CFG=$(cat <<EOF
+{
+  "kv_connector": "MultiConnector",
+  "kv_role": "kv_both",
+  "kv_connector_extra_config": {
+    "connectors": [
+      {"kv_connector": "MooncakeConnector", "kv_role": "kv_both"},
+      {"kv_connector": "LMCacheConnectorV1", "kv_role": "kv_both"}
+    ]
+  }
+}
+EOF
+)
+KV_CFG=$(echo "$KV_CFG" | tr -d '\n')
+
+VLLM_MOONCAKE_BOOTSTRAP_PORT="$BOOTSTRAP_PORT" \
+CUDA_VISIBLE_DEVICES="$GPU_ID" \
+"$PYTHON" -m vllm.entrypoints.openai.api_server \
+    "${COMMON_VLLM_ARGS[@]}" \
+    --kv-transfer-config "$KV_CFG" \
+    > "$LOG_DIR/vllm_stdout.log" 2> "$LOG_DIR/vllm_stderr.log" &
+VLLM_PID=$!
+echo "$VLLM_PID" > "$LOG_DIR/.vllm.pid"
+echo "vLLM PID=$VLLM_PID"
+wait_for_ready 300

microbench/connector_tax/launch/launch_nixl_both.sh

@@ -0,0 +1,21 @@
+#!/bin/bash
+# vLLM + NIXL kv_both. Skip if NIXL not installed.
+set -euo pipefail
+source "$(dirname "${BASH_SOURCE[0]}")/common.sh"
+
+if ! "$PYTHON" -c 'import nixl' 2>/dev/null; then
+    echo "SKIP: nixl module not importable" >&2
+    exit 42  # special skip code
+fi
+
+KV_CFG='{"kv_connector":"NixlConnector","kv_role":"kv_both"}'
+
+CUDA_VISIBLE_DEVICES="$GPU_ID" \
+"$PYTHON" -m vllm.entrypoints.openai.api_server \
+    "${COMMON_VLLM_ARGS[@]}" \
+    --kv-transfer-config "$KV_CFG" \
+    > "$LOG_DIR/vllm_stdout.log" 2> "$LOG_DIR/vllm_stderr.log" &
+VLLM_PID=$!
+echo "$VLLM_PID" > "$LOG_DIR/.vllm.pid"
+echo "vLLM PID=$VLLM_PID"
+wait_for_ready 240

microbench/connector_tax/launch/launch_noop_connector.sh

@@ -0,0 +1,16 @@
+#!/bin/bash
+# vLLM + custom NoOpConnector loaded by dotted path.
+set -euo pipefail
+source "$(dirname "${BASH_SOURCE[0]}")/common.sh"
+
+KV_CFG='{"kv_connector":"NoOpConnector","kv_connector_module_path":"microbench.connector_tax.tools.noop_connector","kv_role":"kv_both"}'
+
+CUDA_VISIBLE_DEVICES="$GPU_ID" \
+"$PYTHON" -m vllm.entrypoints.openai.api_server \
+    "${COMMON_VLLM_ARGS[@]}" \
+    --kv-transfer-config "$KV_CFG" \
+    > "$LOG_DIR/vllm_stdout.log" 2> "$LOG_DIR/vllm_stderr.log" &
+VLLM_PID=$!
+echo "$VLLM_PID" > "$LOG_DIR/.vllm.pid"
+echo "vLLM PID=$VLLM_PID"
+wait_for_ready 240

microbench/connector_tax/launch/launch_plain.sh

@@ -0,0 +1,13 @@
+#!/bin/bash
+# Plain vLLM, no KV transfer connector. The baseline.
+set -euo pipefail
+source "$(dirname "${BASH_SOURCE[0]}")/common.sh"
+
+CUDA_VISIBLE_DEVICES="$GPU_ID" \
+"$PYTHON" -m vllm.entrypoints.openai.api_server \
+    "${COMMON_VLLM_ARGS[@]}" \
+    > "$LOG_DIR/vllm_stdout.log" 2> "$LOG_DIR/vllm_stderr.log" &
+VLLM_PID=$!
+echo "$VLLM_PID" > "$LOG_DIR/.vllm.pid"
+echo "vLLM PID=$VLLM_PID"
+wait_for_ready 240

microbench/connector_tax/metrics_sampler.py

@@ -0,0 +1,110 @@
+#!/usr/bin/env python3
+"""1 Hz /metrics scraper for connector_tax microbench.
+
+Usage:
+  metrics_sampler.py --url http://127.0.0.1:8000/metrics \
+                     --output results/<run>/metrics.jsonl \
+                     --interval 1.0
+"""
+
+import argparse
+import json
+import time
+import urllib.request
+
+
+def parse_prom(text: str) -> dict:
+    """Parse Prometheus text-format metrics. Returns {name: [(labels, value)]}."""
+    out: dict[str, list[tuple[dict[str, str], float]]] = {}
+    for line in text.splitlines():
+        line = line.strip()
+        if not line or line.startswith("#"):
+            continue
+        # name{labels} value [timestamp]
+        if "{" in line:
+            name, rest = line.split("{", 1)
+            labels_str, val_str = rest.rsplit("}", 1)
+            labels = {}
+            for piece in labels_str.split(","):
+                if "=" in piece:
+                    k, v = piece.split("=", 1)
+                    labels[k.strip()] = v.strip().strip('"')
+            try:
+                val = float(val_str.strip().split()[0])
+            except (ValueError, IndexError):
+                continue
+        else:
+            parts = line.split()
+            if len(parts) < 2:
+                continue
+            name = parts[0]
+            try:
+                val = float(parts[1])
+            except ValueError:
+                continue
+            labels = {}
+        out.setdefault(name, []).append((labels, val))
+    return out
+
+
+KEEP_PREFIXES = (
+    "vllm:num_requests_running",
+    "vllm:num_requests_waiting",
+    "vllm:gpu_cache_usage_perc",
+    "vllm:time_to_first_token_seconds",
+    "vllm:time_per_output_token_seconds",
+    "vllm:request_prefill_time_seconds",
+    "vllm:request_decode_time_seconds",
+    "vllm:iteration_tokens_total",
+    "vllm:e2e_request_latency_seconds",
+)
+
+
+def collapse(parsed: dict) -> dict:
+    """Keep only metrics whose names start with one of the prefixes; flatten
+    histogram counts into '_bucket' / '_count' / '_sum' suffix entries."""
+    out = {}
+    for name, entries in parsed.items():
+        if not any(name.startswith(p) for p in KEEP_PREFIXES):
+            continue
+        # Most are scalars (ignore label dimensions for compactness)
+        # For histograms we keep _count/_sum and skip individual buckets
+        if name.endswith("_bucket"):
+            continue
+        # Sum across labels to get a single number
+        total = sum(v for _lbl, v in entries)
+        out[name] = total
+    return out
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--url", required=True,
+                    help="http://host:port/metrics")
+    ap.add_argument("--output", required=True)
+    ap.add_argument("--interval", type=float, default=1.0)
+    ap.add_argument("--duration", type=float, default=0.0,
+                    help="Stop after N seconds; 0 = run until killed")
+    args = ap.parse_args()
+
+    out = open(args.output, "a", buffering=1)
+    t_start = time.time()
+    while True:
+        try:
+            with urllib.request.urlopen(args.url, timeout=2.0) as r:
+                text = r.read().decode("utf-8")
+            parsed = parse_prom(text)
+            sample = collapse(parsed)
+            sample["t_unix"] = time.time()
+            out.write(json.dumps(sample) + "\n")
+        except Exception as e:
+            err = {"t_unix": time.time(), "error": str(e)}
+            out.write(json.dumps(err) + "\n")
+
+        if args.duration > 0 and time.time() - t_start >= args.duration:
+            break
+        time.sleep(args.interval)
+
+
+if __name__ == "__main__":
+    main()

microbench/connector_tax/patches/apply_step_timing.py

@@ -0,0 +1,171 @@
+#!/usr/bin/env python3
+"""Apply / revert step-timing additions to vLLM's existing agentic step log.
+
+vLLM already has a per-step JSONL emitter triggered by AGENTIC_STEP_LOG_PATH.
+This patch enriches it with three duration fields:
+
+  step_duration_us   : full schedule() wall time
+  build_meta_us      : duration of self.connector.build_connector_meta(...)
+  emit_overhead_us   : duration of _agentic_emit_step_log itself
+                       (lets us subtract the patch's own cost)
+
+We also add a worker-side patch in kv_connector_model_runner_mixin.py to
+record start_load_kv() duration into a per-process JSONL file pointed to
+by VLLM_PD_PROFILE_LOG=$RUN_DIR/worker_step.jsonl.
+
+All inserts are marked with `# CONNECTOR_TAX_PATCH` so revert is just
+"delete those lines".
+
+Usage:
+  python apply_step_timing.py --apply  [--vllm-root PATH]
+  python apply_step_timing.py --revert [--vllm-root PATH]
+"""
+
+import argparse
+import re
+import sys
+from pathlib import Path
+
+MARKER = "# CONNECTOR_TAX_PATCH"
+DEFAULT_VLLM_ROOT = (
+    Path.home()
+    / "agentic-kv/.venv/lib/python3.12/site-packages/vllm"
+)
+
+
+def already_patched(text: str) -> bool:
+    return MARKER in text
+
+
+def revert_text(text: str) -> str:
+    out = [l for l in text.splitlines() if MARKER not in l]
+    return "\n".join(out) + ("\n" if text.endswith("\n") else "")
+
+
+# ── scheduler.py patches ────────────────────────────────────────────────────
+def patch_scheduler(text: str) -> str:
+    if already_patched(text):
+        print("  scheduler.py already patched, skipping")
+        return text
+
+    # 1. At top of schedule(): record _step_t0
+    pat = (
+        r"(    def schedule\(self\) -> SchedulerOutput:\n"
+        r"        # NOTE\(woosuk\) on the scheduling algorithm:)"
+    )
+    repl = (
+        r"    def schedule(self) -> SchedulerOutput:\n"
+        r"        import time as _t  " + MARKER + "\n"
+        r"        _step_t0 = _t.perf_counter_ns()  " + MARKER + "\n"
+        r"        self._ct_build_meta_ns = 0  " + MARKER + "\n"
+        r"        # NOTE(woosuk) on the scheduling algorithm:"
+    )
+    text, n = re.subn(pat, repl, text, count=1)
+    if n == 0:
+        raise RuntimeError("Failed to patch schedule() entry")
+
+    # 2. Time the build_connector_meta call
+    pat = (
+        r"        if self\.connector is not None:\n"
+        r"            meta: KVConnectorMetadata = self\.connector\.build_connector_meta\(\n"
+        r"                scheduler_output\n"
+        r"            \)\n"
+        r"            scheduler_output\.kv_connector_metadata = meta"
+    )
+    repl = (
+        "        if self.connector is not None:\n"
+        f"            _bm_t0 = _t.perf_counter_ns()  {MARKER}\n"
+        "            meta: KVConnectorMetadata = self.connector.build_connector_meta(\n"
+        "                scheduler_output\n"
+        "            )\n"
+        f"            self._ct_build_meta_ns = _t.perf_counter_ns() - _bm_t0  {MARKER}\n"
+        "            scheduler_output.kv_connector_metadata = meta"
+    )
+    text, n = re.subn(pat, repl, text, count=1)
+    if n == 0:
+        raise RuntimeError("Failed to patch build_connector_meta")
+
+    # 3. Pass step duration into _agentic_emit_step_log via attribute
+    #    (cleaner than threading kwargs through). Then in the emit
+    #    function inject the new fields into `record`.
+    pat = (
+        r"        if self\._agentic_step_log_fh is not None:\n"
+        r"            self\._agentic_emit_step_log\("
+    )
+    repl = (
+        f"        if self._agentic_step_log_fh is not None:\n"
+        f"            self._ct_step_duration_ns = _t.perf_counter_ns() - _step_t0  {MARKER}\n"
+        f"            self._agentic_emit_step_log("
+    )
+    text, n = re.subn(pat, repl, text, count=1)
+    if n == 0:
+        raise RuntimeError("Failed to patch step_duration insertion")
+
+    # 4. Inject extra fields into the `record` dict in _agentic_emit_step_log
+    pat = (
+        r"        record = \{\n"
+        r"            \"t_unix\": _time\.time\(\),"
+    )
+    repl = (
+        "        record = {\n"
+        f"            \"step_duration_us\": getattr(self, '_ct_step_duration_ns', 0) // 1000,  {MARKER}\n"
+        f"            \"build_meta_us\":    getattr(self, '_ct_build_meta_ns', 0) // 1000,  {MARKER}\n"
+        "            \"t_unix\": _time.time(),"
+    )
+    text, n = re.subn(pat, repl, text, count=1)
+    if n == 0:
+        raise RuntimeError("Failed to patch record dict")
+
+    return text
+
+
+# ── apply / revert driver ───────────────────────────────────────────────────
+def apply_to_file(path: Path, fn) -> bool:
+    if not path.exists():
+        print(f"  SKIP {path} (not found)")
+        return False
+    orig = path.read_text()
+    new = fn(orig)
+    if new == orig:
+        print(f"  unchanged: {path}")
+        return False
+    path.write_text(new)
+    print(f"  patched ({new.count(MARKER)} marks): {path}")
+    return True
+
+
+def revert_file(path: Path) -> bool:
+    if not path.exists():
+        return False
+    orig = path.read_text()
+    new = revert_text(orig)
+    if new == orig:
+        print(f"  no marks: {path}")
+        return False
+    path.write_text(new)
+    print(f"  reverted: {path}")
+    return True
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--apply", action="store_true")
+    ap.add_argument("--revert", action="store_true")
+    ap.add_argument("--vllm-root", type=Path, default=DEFAULT_VLLM_ROOT)
+    args = ap.parse_args()
+
+    if not (args.apply ^ args.revert):
+        ap.error("Specify exactly one of --apply / --revert")
+
+    sched = args.vllm_root / "v1/core/sched/scheduler.py"
+
+    if args.apply:
+        print(f"Applying connector-tax step-timing patch to {args.vllm_root}")
+        apply_to_file(sched, patch_scheduler)
+    else:
+        print(f"Reverting connector-tax step-timing patch from {args.vllm_root}")
+        revert_file(sched)
+
+
+if __name__ == "__main__":
+    main()

microbench/connector_tax/plot_connector_tax.py

@@ -0,0 +1,263 @@
+#!/usr/bin/env python3
+"""Plot Figures 1-5 from connector_tax aggregate.
+
+Requires aggregate.json + aggregate.csv from analyze.py.
+
+Figure 1: TTFT p90 vs send rate, line per config (Phase A)
+Figure 2: TPOT p90 vs send rate
+Figure 3: Achieved throughput vs requested
+Figure 4: Substrate tax bar at ref_safe and ref_load
+Figure 5: Shape-dependent tax heatmap (Phase B)
+"""
+
+import argparse
+import json
+from collections import defaultdict
+from pathlib import Path
+
+import numpy as np
+import matplotlib
+matplotlib.use("Agg")
+import matplotlib.pyplot as plt
+
+
+CONFIG_COLORS = {
+    "plain":                   "#000000",
+    "noop_connector":          "#7f7f7f",
+    "mooncake_producer":       "#1f77b4",
+    "mooncake_consumer":       "#17becf",
+    "mooncake_both":           "#d62728",
+    "nixl_both":               "#ff7f0e",
+    "lmcache_only":            "#2ca02c",
+    "multi_mooncake_lmcache":  "#9467bd",
+}
+
+
+def load(root: Path):
+    agg = json.loads((root / "aggregate.json").read_text())
+    return agg
+
+
+def fig1_2_3(agg, out_dir):
+    configs = agg["configs"]
+
+    # Rates
+    rates = agg["rates_swept"]
+
+    # ----- fig 1: TTFT p90 -----
+    fig, ax = plt.subplots(figsize=(10, 5.5))
+    for cfg, d in configs.items():
+        cells = sorted(d["phase_a"], key=lambda c: c["rate_target"])
+        xs = [c["rate_target"] for c in cells]
+        ys = [c.get("ttft_ms_p90") for c in cells]
+        ax.plot(xs, ys, "o-", label=cfg,
+                color=CONFIG_COLORS.get(cfg, None), linewidth=2)
+        # mark saturation
+        for c in cells:
+            if c.get("saturated"):
+                ax.plot([c["rate_target"]], [c["ttft_ms_p90"]], "x",
+                        markersize=12, mew=2,
+                        color=CONFIG_COLORS.get(cfg, "red"))
+    ax.set_xscale("log", base=2)
+    ax.set_yscale("log")
+    ax.set_xticks(rates)
+    ax.set_xticklabels([str(r) for r in rates])
+    ax.set_xlabel("Send rate (req/s)")
+    ax.set_ylabel("TTFT p90 (ms, log)")
+    ax.set_title("Figure 1 — TTFT p90 vs send rate (Phase A)\n"
+                 "× = saturation criterion fired")
+    ax.grid(True, which="both", linestyle="--", alpha=0.4)
+    ax.legend(fontsize=8, loc="upper left")
+    fig.tight_layout()
+    fig.savefig(out_dir / "fig1_ttft_vs_rate.png", dpi=160)
+    plt.close(fig)
+
+    # ----- fig 2: TPOT p90 -----
+    fig, ax = plt.subplots(figsize=(10, 5.5))
+    for cfg, d in configs.items():
+        cells = sorted(d["phase_a"], key=lambda c: c["rate_target"])
+        xs = [c["rate_target"] for c in cells]
+        ys = [c.get("tpot_ms_p90") for c in cells]
+        ax.plot(xs, ys, "o-", label=cfg,
+                color=CONFIG_COLORS.get(cfg, None), linewidth=2)
+    ax.set_xscale("log", base=2)
+    ax.set_xticks(rates)
+    ax.set_xticklabels([str(r) for r in rates])
+    ax.set_xlabel("Send rate (req/s)")
+    ax.set_ylabel("TPOT p90 (ms)")
+    ax.set_title("Figure 2 — TPOT p90 vs send rate (Phase A)")
+    ax.grid(True, linestyle="--", alpha=0.4)
+    ax.legend(fontsize=8, loc="upper left")
+    fig.tight_layout()
+    fig.savefig(out_dir / "fig2_tpot_vs_rate.png", dpi=160)
+    plt.close(fig)
+
+    # ----- fig 3: throughput -----
+    fig, ax = plt.subplots(figsize=(10, 5.5))
+    max_x = max(rates) if rates else 1
+    ax.plot([0, max_x], [0, max_x], "k--", alpha=0.4, label="ideal y=x")
+    for cfg, d in configs.items():
+        cells = sorted(d["phase_a"], key=lambda c: c["rate_target"])
+        xs = [c["rate_target"] for c in cells]
+        ys = [c.get("throughput_effective_rps") for c in cells]
+        ax.plot(xs, ys, "o-", label=cfg,
+                color=CONFIG_COLORS.get(cfg, None), linewidth=2)
+    ax.set_xlabel("Send rate (req/s)")
+    ax.set_ylabel("Effective throughput (req/s)")
+    ax.set_title("Figure 3 — Throughput tracking (Phase A)")
+    ax.grid(True, linestyle="--", alpha=0.4)
+    ax.legend(fontsize=8, loc="upper left")
+    fig.tight_layout()
+    fig.savefig(out_dir / "fig3_throughput_vs_rate.png", dpi=160)
+    plt.close(fig)
+
+
+def fig4(agg, out_dir):
+    configs = agg["configs"]
+    if "plain" not in configs:
+        return
+    plain = configs["plain"]
+
+    def cell_at(d, r):
+        for c in d["phase_a"]:
+            if abs(c["rate_target"] - r) < 1e-6:
+                return c
+        return None
+
+    def tax(c_cfg, c_plain, key):
+        if c_cfg is None or c_plain is None:
+            return None
+        a, b = c_cfg.get(key), c_plain.get(key)
+        if not a or not b:
+            return None
+        return a / b - 1
+
+    rates_used = []
+    if agg.get("ref_safe") is not None:
+        rates_used.append(("ref_safe", agg["ref_safe"]))
+    if agg.get("ref_load") is not None and agg["ref_load"] != agg.get("ref_safe"):
+        rates_used.append(("ref_load", agg["ref_load"]))
+
+    if not rates_used:
+        return
+
+    cfg_names = [c for c in configs if c != "plain"]
+    fig, axes = plt.subplots(1, 2, figsize=(13, 5))
+    if len(rates_used) == 1:
+        axes = [axes[0]]
+    for ax, (label, r) in zip(axes, rates_used):
+        plain_cell = cell_at(plain, r)
+        ttft_taxes = []
+        tpot_taxes = []
+        for cfg in cfg_names:
+            c = cell_at(configs[cfg], r)
+            ttft_taxes.append(tax(c, plain_cell, "ttft_ms_p90") or 0)
+            tpot_taxes.append(tax(c, plain_cell, "tpot_ms_p90") or 0)
+
+        x = np.arange(len(cfg_names))
+        w = 0.4
+        ax.bar(x - w/2, [v * 100 for v in ttft_taxes], width=w,
+               label="TTFT p90 tax %", color="#d62728", alpha=0.85)
+        ax.bar(x + w/2, [v * 100 for v in tpot_taxes], width=w,
+               label="TPOT p90 tax %", color="#1f77b4", alpha=0.85)
+        ax.axhline(0, color="black", linewidth=0.5)
+        ax.set_xticks(x)
+        ax.set_xticklabels(cfg_names, rotation=30, ha="right", fontsize=8)
+        ax.set_ylabel("Tax vs plain (%)")
+        ax.set_title(f"{label} (rate={r} req/s)")
+        ax.grid(True, axis="y", linestyle="--", alpha=0.4)
+        ax.legend(fontsize=8)
+    fig.suptitle("Figure 4 — Substrate tax (TTFT p90 + TPOT p90) "
+                 "at reference rates", fontweight="bold")
+    fig.tight_layout()
+    fig.savefig(out_dir / "fig4_substrate_tax.png", dpi=160)
+    plt.close(fig)
+
+
+def fig5(agg, out_dir):
+    configs = agg["configs"]
+    if "plain" not in configs:
+        return
+
+    # Build (input, output) → ttft_p90 per config
+    cfg_names = [c for c in configs if c != "plain"]
+
+    def shape_map(d):
+        m = {}
+        for c in d.get("phase_b", []):
+            key = (c["input_tokens"], c["output_tokens"])
+            m[key] = c.get("ttft_ms_p90")
+        return m
+
+    plain_map = shape_map(configs["plain"])
+    if not plain_map:
+        return
+
+    inputs = sorted({k[0] for k in plain_map})
+    outputs = sorted({k[1] for k in plain_map})
+
+    n = len(cfg_names)
+    cols = min(3, n)
+    rows = (n + cols - 1) // cols
+    fig, axes = plt.subplots(rows, cols, figsize=(5 * cols, 4 * rows))
+    if n == 1:
+        axes = np.array([[axes]])
+    elif rows == 1:
+        axes = axes.reshape(1, -1)
+
+    for idx, cfg in enumerate(cfg_names):
+        ax = axes[idx // cols][idx % cols]
+        cmap = shape_map(configs[cfg])
+        mat = np.full((len(outputs), len(inputs)), np.nan)
+        for i, ip in enumerate(inputs):
+            for j, op in enumerate(outputs):
+                a = cmap.get((ip, op))
+                b = plain_map.get((ip, op))
+                if a and b:
+                    mat[j, i] = a / b - 1
+        im = ax.imshow(mat * 100, cmap="YlOrRd", aspect="auto")
+        ax.set_xticks(range(len(inputs)))
+        ax.set_xticklabels([f"{x//1024}k" if x >= 1024 else str(x) for x in inputs])
+        ax.set_yticks(range(len(outputs)))
+        ax.set_yticklabels([str(y) for y in outputs])
+        ax.set_xlabel("input")
+        ax.set_ylabel("output")
+        ax.set_title(cfg, fontsize=10)
+        for i in range(len(inputs)):
+            for j in range(len(outputs)):
+                v = mat[j, i]
+                if not np.isnan(v):
+                    txt = f"{v*100:.0f}%"
+                    ax.text(i, j, txt, ha="center", va="center",
+                            fontsize=9,
+                            color="white" if v * 100 > 30 else "black")
+        plt.colorbar(im, ax=ax, fraction=0.04, pad=0.02)
+
+    # Hide leftover axes
+    for idx in range(n, rows * cols):
+        axes[idx // cols][idx % cols].axis("off")
+
+    fig.suptitle("Figure 5 — TTFT p90 substrate tax (%) by shape (Phase B)",
+                 fontweight="bold")
+    fig.tight_layout()
+    fig.savefig(out_dir / "fig5_shape_tax_heatmap.png", dpi=160)
+    plt.close(fig)
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--root", type=Path, required=True)
+    args = ap.parse_args()
+
+    agg = load(args.root)
+    out = args.root
+    out.mkdir(parents=True, exist_ok=True)
+
+    fig1_2_3(agg, out)
+    fig4(agg, out)
+    fig5(agg, out)
+    print(f"Saved figures into {out}")
+
+
+if __name__ == "__main__":
+    main()

microbench/connector_tax/run_all.sh

@@ -0,0 +1,232 @@
+#!/bin/bash
+# 5-stage barrier orchestrator for connector_tax microbench.
+#
+#   Stage 0 — pre-flight + (optional) step-timing patch
+#   Stage 1 — Phase A (rate sweep) for all configs
+#   Stage 2 — pick reference rates from Phase A
+#   Stage 3 — Phase B (shape sweep at ref_safe) for all configs
+#   Stage 4 — revert patch + analyze + plot
+#
+# Configurable via env:
+#   CT_DATE           : run-id directory tag (default $(date +%Y%m%d_%H%M))
+#   PORT              : vLLM port (default 8000)
+#   GPU_ID            : single GPU index (default 0)
+#   MODEL_PATH        : path to model
+#   PHASE_A_RATES     : default 0.5,1,2,4,8,16,32
+#   PHASE_B_SHAPES    : default 512x64,512x256,512x1024,4096x64,4096x256,4096x1024,32768x64,32768x256,32768x1024
+#   MIN_COMPLETED     : default 200
+#   DURATION          : default 60
+#   SKIP_PATCH        : set to 1 to bypass scheduler patch
+#   STAGES            : space-separated list of stages to run, e.g. "1 3 4"
+#                       defaults to "0 1 2 3 4"
+
+set -uo pipefail
+
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+RESULTS_ROOT="$HERE/results"
+RUN_DATE="${CT_DATE:-$(date +%Y%m%d_%H%M)}"
+RUN_ROOT="$RESULTS_ROOT/$RUN_DATE"
+mkdir -p "$RUN_ROOT"
+
+PORT="${PORT:-8000}"
+GPU_ID="${GPU_ID:-0}"
+MODEL_PATH="${MODEL_PATH:-$HOME/models/Qwen/Qwen3-Coder-30B-A3B-Instruct}"
+PHASE_A_RATES="${PHASE_A_RATES:-0.5,1,2,4,8,16,32}"
+PHASE_B_SHAPES="${PHASE_B_SHAPES:-512x64,512x256,512x1024,4096x64,4096x256,4096x1024,32768x64,32768x256,32768x1024}"
+MIN_COMPLETED="${MIN_COMPLETED:-200}"
+DURATION="${DURATION:-60}"
+STAGES="${STAGES:-0 1 2 3 4}"
+
+ALL_CONFIGS=(plain noop_connector mooncake_producer mooncake_consumer mooncake_both nixl_both lmcache_only multi_mooncake_lmcache)
+
+# Shuffle config order for thermal-drift robustness.
+shuffle_configs() {
+    printf '%s\n' "$@" | shuf
+}
+
+PYTHON="$(cd "$HERE/../.." && pwd)/.venv/bin/python"
+[[ -x "$PYTHON" ]] || PYTHON="$(command -v python3)"
+
+PROJ_DIR="$(cd "$HERE/../.." && pwd)"
+export PYTHONPATH="$PROJ_DIR:${PYTHONPATH:-}"
+
+manifest() {
+    local config="$1" stage="$2" status="$3" note="$4"
+    echo "$(date -Iseconds) | $config | $stage | $status | $note" \
+        >> "$RUN_ROOT/MANIFEST.tsv"
+}
+
+kill_vllm() {
+    local pidfile="$1"
+    if [[ -f "$pidfile" ]]; then
+        local pid; pid=$(cat "$pidfile")
+        if [[ -n "$pid" ]]; then
+            kill -9 "$pid" 2>/dev/null || true
+        fi
+    fi
+    pkill -f "port $PORT" 2>/dev/null || true
+    sleep 5
+    # wait for GPU release
+    for _ in $(seq 1 30); do
+        used=$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits | head -n $((GPU_ID + 1)) | tail -1)
+        if [[ "$used" -lt 1000 ]]; then
+            break
+        fi
+        sleep 2
+    done
+}
+
+run_phase_a() {
+    local config="$1"
+    local cfg_dir="$RUN_ROOT/$config"
+    mkdir -p "$cfg_dir"
+    echo "=== [$config] Phase A ==="
+
+    # Launch
+    RUN_DIR="$cfg_dir" PORT="$PORT" GPU_ID="$GPU_ID" MODEL_PATH="$MODEL_PATH" \
+        bash "$HERE/launch/launch_${config}.sh"
+    local rc=$?
+    if [[ $rc == 42 ]]; then
+        manifest "$config" "phase_a" "SKIP" "dependency missing"
+        return 0
+    fi
+    if [[ $rc != 0 ]]; then
+        manifest "$config" "phase_a" "FAIL" "launch rc=$rc"
+        return 0
+    fi
+
+    # /metrics sampler in background
+    "$PYTHON" "$HERE/metrics_sampler.py" \
+        --url "http://127.0.0.1:$PORT/metrics" \
+        --output "$cfg_dir/metrics_A.jsonl" \
+        --interval 1.0 &
+    local sampler_pid=$!
+
+    # bench loop
+    "$PYTHON" "$HERE/bench_loop.py" \
+        --url "http://127.0.0.1:$PORT/v1/chat/completions" \
+        --model "$MODEL_PATH" \
+        --phase A \
+        --rates "$PHASE_A_RATES" \
+        --shape "4096,256" \
+        --duration "$DURATION" \
+        --min-completed "$MIN_COMPLETED" \
+        --warmup 10 \
+        --output-dir "$cfg_dir"
+    local bench_rc=$?
+
+    kill -9 "$sampler_pid" 2>/dev/null || true
+    kill_vllm "$cfg_dir/.vllm.pid"
+    [[ -f "$cfg_dir/.dummy.pid" ]] && kill -9 "$(cat "$cfg_dir/.dummy.pid")" 2>/dev/null
+
+    if [[ $bench_rc != 0 ]]; then
+        manifest "$config" "phase_a" "FAIL" "bench rc=$bench_rc"
+        return 0
+    fi
+    manifest "$config" "phase_a" "OK" ""
+}
+
+run_phase_b() {
+    local config="$1"
+    local cfg_dir="$RUN_ROOT/$config"
+    mkdir -p "$cfg_dir"
+
+    local ref_safe; ref_safe=$(jq -r '.ref_safe // empty' "$RUN_ROOT/aggregate.json" 2>/dev/null)
+    if [[ -z "$ref_safe" || "$ref_safe" == "null" ]]; then
+        echo "ref_safe not available; skipping Phase B for $config"
+        manifest "$config" "phase_b" "SKIP" "ref_safe undefined"
+        return 0
+    fi
+    echo "=== [$config] Phase B (rate=$ref_safe) ==="
+
+    RUN_DIR="$cfg_dir" PORT="$PORT" GPU_ID="$GPU_ID" MODEL_PATH="$MODEL_PATH" \
+        bash "$HERE/launch/launch_${config}.sh"
+    local rc=$?
+    if [[ $rc == 42 ]]; then
+        manifest "$config" "phase_b" "SKIP" "dependency missing"
+        return 0
+    fi
+    if [[ $rc != 0 ]]; then
+        manifest "$config" "phase_b" "FAIL" "launch rc=$rc"
+        return 0
+    fi
+
+    "$PYTHON" "$HERE/metrics_sampler.py" \
+        --url "http://127.0.0.1:$PORT/metrics" \
+        --output "$cfg_dir/metrics_B.jsonl" \
+        --interval 1.0 &
+    local sampler_pid=$!
+
+    "$PYTHON" "$HERE/bench_loop.py" \
+        --url "http://127.0.0.1:$PORT/v1/chat/completions" \
+        --model "$MODEL_PATH" \
+        --phase B \
+        --rate "$ref_safe" \
+        --shapes "$PHASE_B_SHAPES" \
+        --duration "$DURATION" \
+        --min-completed "$MIN_COMPLETED" \
+        --warmup 10 \
+        --output-dir "$cfg_dir"
+    local bench_rc=$?
+
+    kill -9 "$sampler_pid" 2>/dev/null || true
+    kill_vllm "$cfg_dir/.vllm.pid"
+    [[ -f "$cfg_dir/.dummy.pid" ]] && kill -9 "$(cat "$cfg_dir/.dummy.pid")" 2>/dev/null
+
+    if [[ $bench_rc != 0 ]]; then
+        manifest "$config" "phase_b" "FAIL" "bench rc=$bench_rc"
+        return 0
+    fi
+    manifest "$config" "phase_b" "OK" ""
+}
+
+# ── Stage 0 — pre-flight ────────────────────────────────────────────────
+if [[ " $STAGES " == *" 0 "* ]]; then
+    echo "=== Stage 0 — pre-flight ==="
+    mkdir -p "$RUN_ROOT/preflight"
+    pip freeze > "$RUN_ROOT/preflight/pip_freeze.txt" 2>&1 || true
+    nvidia-smi -L > "$RUN_ROOT/preflight/nvidia.txt" 2>&1 || true
+
+    # Apply step-timing patch unless SKIP_PATCH=1
+    if [[ "${SKIP_PATCH:-0}" != "1" ]]; then
+        if "$PYTHON" "$HERE/patches/apply_step_timing.py" --apply > "$RUN_ROOT/preflight/patch.log" 2>&1; then
+            echo "step_timing_available=true" > "$RUN_ROOT/preflight/patch_status.txt"
+        else
+            echo "step_timing_available=false (apply failed)" > "$RUN_ROOT/preflight/patch_status.txt"
+            cat "$RUN_ROOT/preflight/patch.log"
+        fi
+    else
+        echo "step_timing_available=false (SKIP_PATCH=1)" > "$RUN_ROOT/preflight/patch_status.txt"
+    fi
+fi
+
+# ── Stage 1 — Phase A all configs ──────────────────────────────────────
+if [[ " $STAGES " == *" 1 "* ]]; then
+    echo "=== Stage 1 — Phase A (all configs, randomized) ==="
+    for cfg in $(shuffle_configs "${ALL_CONFIGS[@]}"); do
+        run_phase_a "$cfg"
+    done
+fi
+
+# ── Stage 2 — pick reference rates ─────────────────────────────────────
+if [[ " $STAGES " == *" 2 "* ]]; then
+    echo "=== Stage 2 — pick reference rates ==="
+    "$PYTHON" "$HERE/analyze.py" --root "$RUN_ROOT"
+fi
+
+# ── Stage 3 — Phase B all configs ──────────────────────────────────────
+if [[ " $STAGES " == *" 3 "* ]]; then
+    echo "=== Stage 3 — Phase B (all configs, randomized) ==="
+    for cfg in $(shuffle_configs "${ALL_CONFIGS[@]}"); do
+        run_phase_b "$cfg"
+    done
+fi
+
+# ── Stage 4 — analyze + plot + revert ──────────────────────────────────
+if [[ " $STAGES " == *" 4 "* ]]; then
+    echo "=== Stage 4 — re-analyze with Phase B + plot + revert patch ==="
+    "$PYTHON" "$HERE/analyze.py" --root "$RUN_ROOT"
+    "$PYTHON" "$HERE/plot_connector_tax.py" --root "$RUN_ROOT"
+    "$PYTHON" "$HERE/patches/apply_step_timing.py" --revert >> "$RUN_ROOT/preflight/patch.log" 2>&1 || true
+    echo "Done. Results in $RUN_ROOT"
+fi

microbench/connector_tax/tools/dummy_bootstrap.py

@@ -0,0 +1,84 @@
+"""Dummy Mooncake bootstrap server for kv_consumer pre-flight.
+
+Exposes the same HTTP routes as MooncakeBootstrapServer but returns
+empty / accepting responses. Allows a kv_consumer vLLM to start up
+without a real prefiller behind it.
+
+Usage:
+  python dummy_bootstrap.py --port 8997
+"""
+
+import argparse
+import logging
+import threading
+
+from fastapi import FastAPI, Request
+from fastapi.responses import JSONResponse
+import uvicorn
+
+logging.basicConfig(level=logging.INFO)
+log = logging.getLogger("dummy_bootstrap")
+
+
+def make_app() -> FastAPI:
+    app = FastAPI()
+    state = {"workers": {}, "hash_table": {}}
+
+    @app.post("/register")
+    async def register_worker(req: Request):
+        body = await req.json()
+        log.info("register_worker: %s", body)
+        # Pretend success
+        dp_rank = int(body.get("dp_rank", 0))
+        engine_id = body.get("engine_id", "dummy-engine")
+        state["workers"][dp_rank] = {
+            "engine_id": engine_id,
+            "worker_addr": body.get("worker_addr", {}),
+        }
+        return JSONResponse({"status": "ok"})
+
+    @app.get("/query")
+    async def query():
+        # Return whatever we have. Empty {} is acceptable for the consumer
+        # because no PD-sep request will actually trigger a pull.
+        return JSONResponse(state["workers"])
+
+    @app.post("/query_blocks")
+    async def query_blocks(req: Request):
+        return JSONResponse({"matched_blocks": []})
+
+    @app.post("/unpin_blocks")
+    async def unpin_blocks(req: Request):
+        return JSONResponse({"status": "ok"})
+
+    @app.post("/push_blocks")
+    async def push_blocks(req: Request):
+        return JSONResponse({"status": "ok"})
+
+    @app.post("/estimate_hit")
+    async def estimate_hit(req: Request):
+        return JSONResponse({"hit_tokens": 0})
+
+    @app.get("/health")
+    async def health():
+        return JSONResponse({"status": "ok"})
+
+    return app
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--host", default="127.0.0.1")
+    ap.add_argument("--port", type=int, default=8997)
+    args = ap.parse_args()
+
+    app = make_app()
+    config = uvicorn.Config(app=app, host=args.host, port=args.port,
+                            log_level="info")
+    server = uvicorn.Server(config)
+    log.info("Dummy Mooncake bootstrap listening on %s:%d", args.host, args.port)
+    server.run()
+
+
+if __name__ == "__main__":
+    main()

microbench/connector_tax/tools/noop_connector.py

@@ -0,0 +1,90 @@
+"""Pure no-op KV connector for measuring vLLM v1 framework overhead.
+
+This connector implements every abstract hook of KVConnectorBase_V1 with
+the cheapest possible no-op return. Loaded via:
+
+  --kv-transfer-config '{
+    "kv_connector_module_path":
+        "microbench.connector_tax.tools.noop_connector:NoOpConnector",
+    "kv_role": "kv_both"
+  }'
+
+It does:
+  - no I/O
+  - no per-step cache key walk
+  - no per-layer save/load
+  - no metadata serialization beyond an empty dataclass
+
+So `tax(NoOpConnector) ≈ pure vLLM v1 framework overhead`.
+"""
+
+from typing import TYPE_CHECKING, Any
+
+from vllm.distributed.kv_transfer.kv_connector.v1.base import (
+    KVConnectorBase_V1,
+    KVConnectorMetadata,
+)
+
+if TYPE_CHECKING:
+    import torch
+    from vllm.attention.backends.abstract import AttentionMetadata
+    from vllm.forward_context import ForwardContext
+    from vllm.v1.core.kv_cache_manager import KVCacheBlocks
+    from vllm.v1.core.sched.output import SchedulerOutput
+    from vllm.v1.request import Request
+
+
+class NoOpConnector(KVConnectorBase_V1):
+    """Empty connector — every hook is a no-op.
+
+    Used as a control to isolate vLLM v1 framework dispatch cost
+    (build_connector_meta walking SchedulerOutput, mixin hooks, etc.)
+    from any specific connector implementation work (RDMA setup,
+    per-layer save, hash table walks).
+    """
+
+    # ---- scheduler-side abstract methods ------------------------------
+    def get_num_new_matched_tokens(
+        self,
+        request: "Request",
+        num_computed_tokens: int,
+    ) -> tuple[int | None, bool]:
+        # Never advertises any external cache hits.
+        return 0, False
+
+    def update_state_after_alloc(
+        self,
+        request: "Request",
+        blocks: "KVCacheBlocks",
+        num_external_tokens: int,
+    ) -> None:
+        return None
+
+    def build_connector_meta(
+        self,
+        scheduler_output: "SchedulerOutput",
+    ) -> KVConnectorMetadata:
+        return KVConnectorMetadata()
+
+    # ---- worker-side abstract methods ---------------------------------
+    def start_load_kv(
+        self,
+        forward_context: "ForwardContext",
+        **kwargs: Any,
+    ) -> None:
+        return None
+
+    def wait_for_layer_load(self, layer_name: str) -> None:
+        return None
+
+    def save_kv_layer(
+        self,
+        layer_name: str,
+        kv_layer: "torch.Tensor",
+        attn_metadata: "AttentionMetadata",
+        **kwargs: Any,
+    ) -> None:
+        return None
+
+    def wait_for_save(self) -> None:
+        return None

microbench/connector_tax/tools/verify_kv_consumer.sh

@@ -0,0 +1,33 @@
+#!/bin/bash
+# Pre-flight: verify Mooncake kv_consumer + dummy bootstrap can start
+# and answer at least a trivial request.
+set -euo pipefail
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+RUN_DIR="${RUN_DIR:-$HERE/../results/preflight/kv_consumer}"
+mkdir -p "$RUN_DIR"
+
+DUMMY_BOOTSTRAP_PORT="${DUMMY_BOOTSTRAP_PORT:-8997}"
+PORT="${PORT:-8000}"
+
+bash "$HERE/../launch/launch_mooncake_consumer.sh" || {
+    echo "SKIP: kv_consumer launch failed (likely Mooncake bootstrap incompatible with dummy)" >&2
+    exit 42
+}
+
+# kv_consumer can be sent a regular (non-PD-sep) request — it will just
+# do local prefill+decode. It should succeed.
+MODEL="$HOME/models/Qwen/$(ls $HOME/models/Qwen | grep Qwen3-Coder-30B | head -1)"
+
+for i in 1 2 3 4 5; do
+    code=$(curl -s -o "$RUN_DIR/req_$i.json" -w "%{http_code}" \
+        -X POST "http://127.0.0.1:$PORT/v1/chat/completions" \
+        -H 'Content-Type: application/json' \
+        -d "{\"model\":\"$MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"hi $i\"}],\"max_tokens\":4,\"temperature\":0}" \
+        --max-time 30 || echo "000")
+    if [[ "$code" != "200" ]]; then
+        echo "FAIL: req $i status=$code" >&2
+        exit 1
+    fi
+done
+
+echo "OK: kv_consumer reachable, all 5 requests succeeded"

microbench/connector_tax/tools/verify_multi_connector.sh

@@ -0,0 +1,35 @@
+#!/bin/bash
+# Pre-flight: verify MultiConnector(Mooncake kv_both, LMCacheConnectorV1).
+# Exits 42 if LMCache missing, 1 on crash, 0 on OK.
+set -euo pipefail
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+RUN_DIR="${RUN_DIR:-$HERE/../results/preflight/multi}"
+mkdir -p "$RUN_DIR"
+
+PORT="${PORT:-8000}"
+
+if ! "$(dirname "$HERE")/launch/launch_multi_mooncake_lmcache.sh"; then
+    rc=$?
+    if [[ $rc == 42 ]]; then
+        echo "SKIP: lmcache missing" >&2
+        exit 42
+    fi
+    echo "FAIL: launch failed with code $rc" >&2
+    exit 1
+fi
+
+MODEL="$HOME/models/Qwen/$(ls $HOME/models/Qwen | grep Qwen3-Coder-30B | head -1)"
+
+for i in 1 2 3 4 5; do
+    code=$(curl -s -o "$RUN_DIR/req_$i.json" -w "%{http_code}" \
+        -X POST "http://127.0.0.1:$PORT/v1/chat/completions" \
+        -H 'Content-Type: application/json' \
+        -d "{\"model\":\"$MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"multi $i\"}],\"max_tokens\":32,\"temperature\":0}" \
+        --max-time 60 || echo "000")
+    if [[ "$code" != "200" ]]; then
+        echo "FAIL: req $i status=$code" >&2
+        exit 1
+    fi
+done
+
+echo "OK: MultiConnector reachable, all 5 requests succeeded"

microbench/connector_tax/tools/verify_noop_connector.sh

@@ -0,0 +1,26 @@
+#!/bin/bash
+# Pre-flight: verify NoOpConnector loads and serves requests.
+# Exits 0 = OK, 42 = SKIP (dependency missing), nonzero = fail.
+
+set -euo pipefail
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+RUN_DIR="${RUN_DIR:-$HERE/../results/preflight/noop}"
+mkdir -p "$RUN_DIR"
+
+bash "$HERE/../launch/launch_noop_connector.sh"
+PORT="${PORT:-8000}"
+
+# Issue 5 short requests
+for i in 1 2 3 4 5; do
+    code=$(curl -s -o "$RUN_DIR/req_$i.json" -w "%{http_code}" \
+        -X POST "http://127.0.0.1:$PORT/v1/chat/completions" \
+        -H 'Content-Type: application/json' \
+        -d "{\"model\":\"$(ls $HOME/models/Qwen | grep Qwen3-Coder-30B | head -1 | xargs -I{} echo $HOME/models/Qwen/{})\",\"messages\":[{\"role\":\"user\",\"content\":\"hello $i\"}],\"max_tokens\":4,\"temperature\":0}" \
+        --max-time 60 || echo "000")
+    if [[ "$code" != "200" ]]; then
+        echo "FAIL: req $i status=$code" >&2
+        exit 1
+    fi
+done
+
+echo "OK: all 5 requests succeeded"

scripts/b3_isolated_policy.sh

@@ -13,7 +13,7 @@ set -euo pipefail
 
 ROOT="${ROOT:-$(cd "$(dirname "$0")/.." && pwd)}"
 VENV="$ROOT/.venv/bin"
-MODEL="${MODEL:-$ROOT/models/Qwen/Qwen3-Coder-30B-A3B-Instruct}"
+MODEL="${MODEL:-$HOME/models/Qwen/Qwen3-Coder-30B-A3B-Instruct}"
 PROXY_PORT="${PROXY_PORT:-9300}"
 BASE_PORT="${BASE_PORT:-8000}"
 GPU_INDICES="${GPU_INDICES:-0 1 2 3 4 5 6 7}"

scripts/b3_sweep.sh

@@ -16,7 +16,7 @@ set -euo pipefail
 
 ROOT="${ROOT:-$(cd "$(dirname "$0")/.." && pwd)}"
 VENV="$ROOT/.venv/bin"
-MODEL="${MODEL:-$ROOT/models/Qwen/Qwen3-Coder-30B-A3B-Instruct}"
+MODEL="${MODEL:-$HOME/models/Qwen/Qwen3-Coder-30B-A3B-Instruct}"
 TRACE="${TRACE:-$ROOT/traces/w600_r0.0015_st30.jsonl}"
 OUTDIR="${OUTDIR:-$ROOT/outputs/b3_sweep_$(date +%Y%m%d_%H%M%S)}"
 PROXY_PORT="${PROXY_PORT:-9300}"

scripts/launch_elastic_p2p.sh

@@ -27,7 +27,7 @@ PROXY_PORT=9090
 HEAVY_THRESHOLD="${HEAVY_THRESHOLD:-20000}"
 MAX_OFFLOAD="${MAX_OFFLOAD:-4}"
 
-trap 'echo "Cleaning up..."; kill $(jobs -p) 2>/dev/null; pkill -9 -f "vllm serve" 2>/dev/null; pkill -9 -f "EngineCore" 2>/dev/null; pkill -9 -f cache_aware_proxy 2>/dev/null; wait 2>/dev/null' EXIT INT TERM
+trap 'echo "Cleaning up..."; kill $(jobs -p) 2>/dev/null || true; pkill -9 -f "vllm serve" 2>/dev/null || true; pkill -9 -f "EngineCore" 2>/dev/null || true; pkill -9 -f cache_aware_proxy 2>/dev/null || true; wait 2>/dev/null || true' EXIT INT TERM
 
 echo "=== Elastic P2P Offload (${N_INSTANCES}× TP=1 kv_both) ==="
 echo "  Model:           $MODEL"

uv.lock

@@ -11,9 +11,11 @@ name = "agentic-kv"
 version = "0.1.0"
 source = { editable = "." }
 dependencies = [
+    { name = "fastapi" },
     { name = "httpx" },
     { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
     { name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
+    { name = "uvicorn" },
 ]
 
 [package.optional-dependencies]
@@ -23,12 +25,32 @@ dev = [
 
 [package.metadata]
 requires-dist = [
+    { name = "fastapi", specifier = ">=0.110" },
     { name = "httpx", specifier = ">=0.27" },
     { name = "numpy", specifier = ">=1.24" },
     { name = "pytest", marker = "extra == 'dev'" },
+    { name = "uvicorn", specifier = ">=0.29" },
 ]
 provides-extras = ["dev"]
 
+[[package]]
+name = "annotated-doc"
+version = "0.0.4"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/57/ba/046ceea27344560984e26a590f90bc7f4a75b06701f653222458922b558c/annotated_doc-0.0.4.tar.gz", hash = "sha256:fbcda96e87e9c92ad167c2e53839e57503ecfda18804ea28102353485033faa4", size = 7288, upload-time = "2025-11-10T22:07:42.062Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/1e/d3/26bf1008eb3d2daa8ef4cacc7f3bfdc11818d111f7e2d0201bc6e3b49d45/annotated_doc-0.0.4-py3-none-any.whl", hash = "sha256:571ac1dc6991c450b25a9c2d84a3705e2ae7a53467b5d111c24fa8baabbed320", size = 5303, upload-time = "2025-11-10T22:07:40.673Z" },
+]
+
+[[package]]
+name = "annotated-types"
+version = "0.7.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/ee/67/531ea369ba64dcff5ec9c3402f9f51bf748cec26dde048a2f973a4eea7f5/annotated_types-0.7.0.tar.gz", hash = "sha256:aff07c09a53a08bc8cfccb9c85b05f1aa9a2a6f23728d790723543408344ce89", size = 16081, upload-time = "2024-05-20T21:33:25.928Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/78/b6/6307fbef88d9b5ee7421e68d78a9f162e0da4900bc5f5793f6d3d0e34fb8/annotated_types-0.7.0-py3-none-any.whl", hash = "sha256:1f02e8b43a8fbbc3f3e0d4f0f4bfc8131bcb4eebe8849b8e5c773f3a1c582a53", size = 13643, upload-time = "2024-05-20T21:33:24.1Z" },
+]
+
 [[package]]
 name = "anyio"
 version = "4.13.0"
@@ -52,6 +74,18 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/59/8c/57e832b7af6d7c5abe66eb3fbe3a3a32f4d11ea23a1aa7131371035be991/certifi-2026.5.20-py3-none-any.whl", hash = "sha256:3c52e209ba0a4ad7aebe60436a4ab349c39e1e602e8c134221e546902ad25897", size = 134134, upload-time = "2026-05-20T11:46:48.578Z" },
 ]
 
+[[package]]
+name = "click"
+version = "8.4.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "colorama", marker = "sys_platform == 'win32'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/9b/98/518d8e5081007684232226f475082b30087d0f585e8457db087298259f49/click-8.4.1.tar.gz", hash = "sha256:918b5633eddf6b41c32d4f454bf0de810065c74e3f7dbf8ee5452f8be88d3e96", size = 353007, upload-time = "2026-05-22T04:08:37.769Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/c7/0d/67e5b4109ea4a837e80daa87c2c696711955e40449a97e8926672534def2/click-8.4.1-py3-none-any.whl", hash = "sha256:482be17c6991b8c19c5429a1e995d9b0efdbb63172824c41f99965dc0ade8ec2", size = 116639, upload-time = "2026-05-22T04:08:35.26Z" },
+]
+
 [[package]]
 name = "colorama"
 version = "0.4.6"
@@ -73,6 +107,22 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/8a/0e/97c33bf5009bdbac74fd2beace167cab3f978feb69cc36f1ef79360d6c4e/exceptiongroup-1.3.1-py3-none-any.whl", hash = "sha256:a7a39a3bd276781e98394987d3a5701d0c4edffb633bb7a5144577f82c773598", size = 16740, upload-time = "2025-11-21T23:01:53.443Z" },
 ]
 
+[[package]]
+name = "fastapi"
+version = "0.136.3"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "annotated-doc" },
+    { name = "pydantic" },
+    { name = "starlette" },
+    { name = "typing-extensions" },
+    { name = "typing-inspection" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/81/2d/ff8d91d7b564d464629a0fd50a4489c97fcb836ac230bf3a7269232a9b1f/fastapi-0.136.3.tar.gz", hash = "sha256:e487fae93ad408e6f47641ee4dfe389864fd7bec92e547ea8498fc13f43e83ab", size = 396410, upload-time = "2026-05-23T18:53:15.192Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/e0/82/45359b62a067409bd929ae8a56b8ed13e5a8c8a61194b3c236920999ab83/fastapi-0.136.3-py3-none-any.whl", hash = "sha256:3d2a69bdf04b7e9f3afa292c3bc7a98816bbfafa10bc9b45f3f3700d2f761620", size = 117481, upload-time = "2026-05-23T18:53:16.924Z" },
+]
+
 [[package]]
 name = "h11"
 version = "0.16.0"
@@ -293,6 +343,137 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },
 ]
 
+[[package]]
+name = "pydantic"
+version = "2.13.4"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "annotated-types" },
+    { name = "pydantic-core" },
+    { name = "typing-extensions" },
+    { name = "typing-inspection" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/18/a5/b60d21ac674192f8ab0ba4e9fd860690f9b4a6e51ca5df118733b487d8d6/pydantic-2.13.4.tar.gz", hash = "sha256:c40756b57adaa8b1efeeced5c196f3f3b7c435f90e84ea7f443901bec8099ef6", size = 844775, upload-time = "2026-05-06T13:43:05.343Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/fd/7b/122376b1fd3c62c1ed9dc80c931ace4844b3c55407b6fb2d199377c9736f/pydantic-2.13.4-py3-none-any.whl", hash = "sha256:45a282cde31d808236fd7ea9d919b128653c8b38b393d1c4ab335c62924d9aba", size = 472262, upload-time = "2026-05-06T13:43:02.641Z" },
+]
+
+[[package]]
+name = "pydantic-core"
+version = "2.46.4"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/9d/56/921726b776ace8d8f5db44c4ef961006580d91dc52b803c489fafd1aa249/pydantic_core-2.46.4.tar.gz", hash = "sha256:62f875393d7f270851f20523dd2e29f082bcc82292d66db2b64ea71f64b6e1c1", size = 471464, upload-time = "2026-05-06T13:37:06.98Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/e7/08/f1ba952f1c8ae5581c70fa9c6da89f247b83e3dd8c09c035d5d7931fc23d/pydantic_core-2.46.4-cp310-cp310-macosx_10_12_x86_64.whl", hash = "sha256:a396dcc17e5a0b164dbe026896245a4fa9ff402edca1dff0be3d53a517f74de4", size = 2113146, upload-time = "2026-05-06T13:37:36.537Z" },
+    { url = "https://files.pythonhosted.org/packages/56/c6/65f646c7ff09bd257f660434adb45c4dfcbbcebcc030562fecf6f5bf887d/pydantic_core-2.46.4-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:da4b951fe36dc7c3a1ccb4e3cd1747c3542b8c9ceede8fc86cae054e764485f5", size = 1949769, upload-time = "2026-05-06T13:37:46.365Z" },
+    { url = "https://files.pythonhosted.org/packages/64/ba/bfb1d928fd5b49e1258935ff104ae356e9fd89384a55bf9f847e9193ad40/pydantic_core-2.46.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:bb63e0198ca18aad131c089b9204c23079c3afa95487e561f4c522d519e55aba", size = 1974958, upload-time = "2026-05-06T13:37:28.611Z" },
+    { url = "https://files.pythonhosted.org/packages/4e/74/76223bfb117b64af743c9b6670d1364516f5c0604f96b48f3272f6af6cc6/pydantic_core-2.46.4-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:f47286a97f0bc9b8859519809077b91b2cefe4ae47fcbf5e466a009c1c5d742b", size = 2042118, upload-time = "2026-05-06T13:36:55.216Z" },
+    { url = "https://files.pythonhosted.org/packages/cb/7b/848732968bc8f48f3187542f08358b9d842db564147b256669426ebb1652/pydantic_core-2.46.4-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:905a0ed8ea6f2d61c1738835f99b699348d7857379083e5fc497fa0c967a407c", size = 2222876, upload-time = "2026-05-06T13:38:25.455Z" },
+    { url = "https://files.pythonhosted.org/packages/b5/2f/e90b63ee2e14bd8d3db8f705a6d75d64e6ee1b7c2c8833747ce706e1e0ce/pydantic_core-2.46.4-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:ea793e075b70290d89d8142074262885d3f7da19634845135751bd6344f73b50", size = 2286703, upload-time = "2026-05-06T13:37:53.304Z" },
+    { url = "https://files.pythonhosted.org/packages/ba/1e/acc4d70f88a0a277e4a1fa77ebb985ceabaf900430f875bf9338e11c9420/pydantic_core-2.46.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:395aebd9183f9d112f569aeb5b2214d1a10a33bec8456447f7fbdfa51d38d4cd", size = 2092042, upload-time = "2026-05-06T13:38:46.981Z" },
+    { url = "https://files.pythonhosted.org/packages/a9/da/0a422b57bf8504102bf3c4ccea9c41bab5a5cee6a54650acf8faf67f5a24/pydantic_core-2.46.4-cp310-cp310-manylinux_2_31_riscv64.whl", hash = "sha256:b078afbc25f3a1436c7a1d2cd3e322497ee99615ba97c563566fdf46aff1ee01", size = 2117231, upload-time = "2026-05-06T13:39:23.146Z" },
+    { url = "https://files.pythonhosted.org/packages/bd/2a/2ac13c3af305843e23c5078c53d135656b3f05a2fd78cb7bbbb12e97b473/pydantic_core-2.46.4-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:f747929cf940cddb5b3668a390056ddd5ba2e5010615ea2dcf4f9c4f3ab8791d", size = 2168388, upload-time = "2026-05-06T13:40:08.06Z" },
+    { url = "https://files.pythonhosted.org/packages/72/04/2beacf7e1607e93eefe4aed1b4709f079b905fb77530179d4f7c71745f22/pydantic_core-2.46.4-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:daa27d92c36f24388fe3ad306b174781c747627f134452e4f128ea00ce1fe8c4", size = 2184769, upload-time = "2026-05-06T13:38:13.901Z" },
+    { url = "https://files.pythonhosted.org/packages/9e/29/d2b9fd9f539133548eaf622c06a4ce176cb46ac59f32d0359c4abc0de047/pydantic_core-2.46.4-cp310-cp310-musllinux_1_1_armv7l.whl", hash = "sha256:19e51f073cd3df251856a8a4189fbdf1de4012c3ebacfb1884f94f1eb406079f", size = 2319312, upload-time = "2026-05-06T13:39:08.24Z" },
+    { url = "https://files.pythonhosted.org/packages/7c/af/0f7a5b85fec6075bea96e3ef9187de38fccced0de92c1e7feda8d5cc7bb9/pydantic_core-2.46.4-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:c1747f85cee84c26985853c6f3d9bd3e75da5212912443fa111c113b9c246f39", size = 2361817, upload-time = "2026-05-06T13:38:43.2Z" },
+    { url = "https://files.pythonhosted.org/packages/25/a4/73363fec545fd3ec025490bdda2743c56d0dd5b6266b1a53bbe9e4265375/pydantic_core-2.46.4-cp310-cp310-win32.whl", hash = "sha256:2f84c03c8607173d16b5a854ec68a2f9079ae03237a54fb506d13af47e1d018d", size = 1987085, upload-time = "2026-05-06T13:39:25.497Z" },
+    { url = "https://files.pythonhosted.org/packages/01/aa/62f082da2c91fac1c234bc9ee0066257ce83f0604abd72e4c9d5991f2d84/pydantic_core-2.46.4-cp310-cp310-win_amd64.whl", hash = "sha256:8358a950c8909158e3df31538a7e4edc2d7265a7c54b47f0864d9e5bae9dcebf", size = 2074311, upload-time = "2026-05-06T13:39:59.922Z" },
+    { url = "https://files.pythonhosted.org/packages/5c/fa/6d7708d2cfc1a832acb6aeb0cd16e801902df8a0f583bb3b4b527fde022e/pydantic_core-2.46.4-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:0e96592440881c74a213e5ad528e2b24d3d4f940de2766bed9010ab1d9e51594", size = 2111872, upload-time = "2026-05-06T13:40:27.596Z" },
+    { url = "https://files.pythonhosted.org/packages/ae/6f/aa064a3e74b5745afbdf250594f38e7ead05e2d651bcb35994b9417a0d4d/pydantic_core-2.46.4-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:e0d65b8c354be7fb5f720c3caa8bc940bc2d20ce749c8e06135f07f8ed95dd7c", size = 1948255, upload-time = "2026-05-06T13:39:12.574Z" },
+    { url = "https://files.pythonhosted.org/packages/43/3a/41114a9f7569b84b4d84e7a018c57c56347dac30c0d4a872946ec4e36c46/pydantic_core-2.46.4-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7bfb192b3f4b9e8a89b6277b6ce787564f62cfd272055f6e685726b111dc7826", size = 1972827, upload-time = "2026-05-06T13:38:19.841Z" },
+    { url = "https://files.pythonhosted.org/packages/ef/25/1ab42e8048fe551934d9884e8d64daa7e990ad386f310a15981aeb6a5b08/pydantic_core-2.46.4-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:9037063db01f09b09e237c282b6792bd4da634b5402c4e7f0c61effed7701a04", size = 2041051, upload-time = "2026-05-06T13:38:10.447Z" },
+    { url = "https://files.pythonhosted.org/packages/94/c2/1a934597ddf08da410385b3b7aae91956a5a76c635effef456074fad7e88/pydantic_core-2.46.4-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:fc010ab034c8c7452522748bf937df58020d256ccae0874463d1f4d01758af8e", size = 2221314, upload-time = "2026-05-06T13:40:13.089Z" },
+    { url = "https://files.pythonhosted.org/packages/02/6d/9e8ad178c9c4df27ad3c8f25d1fe2a7ab0d2ba0559fad4aee5d3d1f16771/pydantic_core-2.46.4-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:8c5dac79fa1614d1e06ca695109c6105923bd9c7d1d6c918d4e637b7e6b32fd3", size = 2285146, upload-time = "2026-05-06T13:38:59.224Z" },
+    { url = "https://files.pythonhosted.org/packages/80/50/540cd3aeefc041beb111125c4bff779831a2111fc6b15a9138cda277d32c/pydantic_core-2.46.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f9fa868638bf362d3d138ea55829cefb3d5f4b0d7f142234382a15e2485dbec4", size = 2089685, upload-time = "2026-05-06T13:38:17.762Z" },
+    { url = "https://files.pythonhosted.org/packages/6b/a4/b440ad35f05f6a38f89fa0f149accb3f0e02be94ca5e15f3c449a61b4bc9/pydantic_core-2.46.4-cp311-cp311-manylinux_2_31_riscv64.whl", hash = "sha256:17299feefe090f2caa5b8e37222bb5f663e4935a8bfa6931d4102e5df1a9f398", size = 2115420, upload-time = "2026-05-06T13:37:58.195Z" },
+    { url = "https://files.pythonhosted.org/packages/99/61/de4f55db8dfd57bfdfa9a12ec90fe1b57c4f41062f7ca86f08586b3e0ac0/pydantic_core-2.46.4-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:4c63ebc82684aa89d9a3bcbd13d515b3be44250dc68dd3bd81526c1cb31286c3", size = 2165122, upload-time = "2026-05-06T13:37:01.167Z" },
+    { url = "https://files.pythonhosted.org/packages/f7/52/7c529d7bdb2d1068bd52f51fe32572c8301f9a4febf1948f10639f1436f5/pydantic_core-2.46.4-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:aaa2a54443eff1950ba5ddc6b6ccda0d9c84a364276a62f969bdf2a390650848", size = 2182573, upload-time = "2026-05-06T13:38:45.04Z" },
+    { url = "https://files.pythonhosted.org/packages/37/b3/7c40325848ba78247f2812dcf9c7274e38cd801820ca6dd9fe63bcfb0eb4/pydantic_core-2.46.4-cp311-cp311-musllinux_1_1_armv7l.whl", hash = "sha256:18e5ceec2ab67e6d5f1a9085e5a24c9c4e2ac4545730bfe668680bca05e555f3", size = 2317139, upload-time = "2026-05-06T13:37:15.539Z" },
+    { url = "https://files.pythonhosted.org/packages/d9/37/f913f81a657c865b75da6c0dbed79876073c2a43b5bd9edbe8da785e4d49/pydantic_core-2.46.4-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:a0f62d0a58f4e7da165457e995725421e0064f2255d8eccebc49f41bbc23b109", size = 2360433, upload-time = "2026-05-06T13:37:30.099Z" },
+    { url = "https://files.pythonhosted.org/packages/c4/67/6acaa1be2567f9256b056d8477158cac7240813956ce86e49deae8e173b4/pydantic_core-2.46.4-cp311-cp311-win32.whl", hash = "sha256:041bde0a48fd37cf71cab1c9d56d3e8625a3793fef1f7dd232b3ff37e978ecda", size = 1985513, upload-time = "2026-05-06T13:38:15.669Z" },
+    { url = "https://files.pythonhosted.org/packages/aa/e6/c505f83dfeda9a2e5c995cfd872949e4d05e12f7feb3dca72f633daefa94/pydantic_core-2.46.4-cp311-cp311-win_amd64.whl", hash = "sha256:6f2eeda33a839975441c86a4119e1383c50b47faf0cbb5176985565c6bb02c33", size = 2071114, upload-time = "2026-05-06T13:40:35.416Z" },
+    { url = "https://files.pythonhosted.org/packages/0f/da/7a263a96d965d9d0df5e8de8a475f33495451117035b09acb110288c381f/pydantic_core-2.46.4-cp311-cp311-win_arm64.whl", hash = "sha256:14f4c5d6db102bd796a627bbb3a17b4cf4574b9ae861d8b7c9a9661c6dd3362d", size = 2044298, upload-time = "2026-05-06T13:38:29.754Z" },
+    { url = "https://files.pythonhosted.org/packages/ce/8c/af022f0af448d7747c5154288d46b5f2bc5f17366eaa0e23e9aa04d59f3b/pydantic_core-2.46.4-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:3245406455a5d98187ec35530fd772b1d799b26667980872c8d4614991e2c4a2", size = 2106158, upload-time = "2026-05-06T13:38:57.215Z" },
+    { url = "https://files.pythonhosted.org/packages/19/95/6195171e385007300f0f5574592e467c568becce2d937a0b6804f218bc49/pydantic_core-2.46.4-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:962ccbab7b642487b1d8b7df90ef677e03134cf1fd8880bf698649b22a69371f", size = 1951724, upload-time = "2026-05-06T13:37:02.697Z" },
+    { url = "https://files.pythonhosted.org/packages/8e/bc/f47d1ff9cbb1620e1b5b697eef06010035735f07820180e74178226b27b3/pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8233f2947cf85404441fd7e0085f53b10c93e0ee78611099b5c7237e36aacbf7", size = 1975742, upload-time = "2026-05-06T13:37:09.448Z" },
+    { url = "https://files.pythonhosted.org/packages/5b/11/9b9a5b0306345664a2da6410877af6e8082481b5884b3ddd78d47c6013ce/pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:3a233125ac121aa3ffba9a2b59edfc4a985a76092dc8279586ab4b71390875e7", size = 2052418, upload-time = "2026-05-06T13:37:38.234Z" },
+    { url = "https://files.pythonhosted.org/packages/f1/b7/a65fec226f5d78fc39f4a13c4cc0c768c22b113438f60c14adc9d2865038/pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5b712b53160b79a5850310b912a5ef8e57e56947c8ad690c227f5c9d7e561712", size = 2232274, upload-time = "2026-05-06T13:38:27.753Z" },
+    { url = "https://files.pythonhosted.org/packages/68/f0/92039db98b907ef49269a8271f67db9cb78ae2fc68062ef7e4e77adb5f61/pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:9401557acd873c3a7f3eb9383edef8ac4968f9510e340f4808d427e75667e7b4", size = 2309940, upload-time = "2026-05-06T13:38:05.353Z" },
+    { url = "https://files.pythonhosted.org/packages/5f/97/2aab507d3d00ca626e8e57c1eac6a79e4e5fbcc63eb99733ff55d1717f65/pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:926c9541b14b12b1681dca8a0b75feb510b06c6341b70a8e500c2fdcff837cce", size = 2094516, upload-time = "2026-05-06T13:39:10.577Z" },
+    { url = "https://files.pythonhosted.org/packages/22/37/a8aca44d40d737dde2bc05b3c6c07dff0de07ce6f82e9f3167aeaf4d5dea/pydantic_core-2.46.4-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:56cb4851bcaf3d117eddcef4fe66afd750a50274b0da8e22be256d10e5611987", size = 2136854, upload-time = "2026-05-06T13:40:22.59Z" },
+    { url = "https://files.pythonhosted.org/packages/24/99/fcef1b79238c06a8cbec70819ac722ba76e02bc8ada9b0fd66eba40da01b/pydantic_core-2.46.4-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:c68fcd102d71ea85c5b2dfac3f4f8476eff42a9e078fd5faefff6d145063536b", size = 2180306, upload-time = "2026-05-06T13:40:10.666Z" },
+    { url = "https://files.pythonhosted.org/packages/ae/6c/fc44000918855b42779d007ae63b0532794739027b2f417321cddbc44f6a/pydantic_core-2.46.4-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:b2f69dec1725e79a012d920df1707de5caf7ed5e08f3be4435e25803efc47458", size = 2190044, upload-time = "2026-05-06T13:40:43.231Z" },
+    { url = "https://files.pythonhosted.org/packages/6b/65/d9cadc9f1920d7a127ad2edba16c1db7916e59719285cd6c94600b0080ba/pydantic_core-2.46.4-cp312-cp312-musllinux_1_1_armv7l.whl", hash = "sha256:8d0820e8192167f80d88d64038e609c31452eeca865b4e1d9950a27a4609b00b", size = 2329133, upload-time = "2026-05-06T13:39:57.365Z" },
+    { url = "https://files.pythonhosted.org/packages/d0/cf/c873d91679f3a30bcf5e7ac280ce5573483e72295307685120d0d5ad3416/pydantic_core-2.46.4-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:fbdb89b3e1c94a30cc5edfce477c6e6a5dc4d8f84665b455c27582f211a1c72c", size = 2374464, upload-time = "2026-05-06T13:38:06.976Z" },
+    { url = "https://files.pythonhosted.org/packages/47/bd/6f2fc8188f31bf10590f1e98e7b306336161fac930a8c514cd7bd828c7dc/pydantic_core-2.46.4-cp312-cp312-win32.whl", hash = "sha256:9aa768456404a8bf48a4406685ac2bec8e72b62c69313734fa3b73cf33b3a894", size = 1974823, upload-time = "2026-05-06T13:40:47.985Z" },
+    { url = "https://files.pythonhosted.org/packages/40/8c/985c1d41ea1107c2534abd9870e4ed5c8e7669b5c308297835c001e7a1c4/pydantic_core-2.46.4-cp312-cp312-win_amd64.whl", hash = "sha256:e9c26f834c65f5752f3f06cb08cb86a913ceb7274d0db6e267808a708b46bc89", size = 2072919, upload-time = "2026-05-06T13:39:21.153Z" },
+    { url = "https://files.pythonhosted.org/packages/c4/ba/f463d006e0c47373ca7ec5e1a261c59dc01ef4d62b2657af925fb0deee3a/pydantic_core-2.46.4-cp312-cp312-win_arm64.whl", hash = "sha256:4fc73cb559bdb54b1134a706a2802a4cddd27a0633f5abb7e53056268751ac6a", size = 2027604, upload-time = "2026-05-06T13:39:03.753Z" },
+    { url = "https://files.pythonhosted.org/packages/51/a2/5d30b469c5267a17b39dec53208222f76a8d351dfac4af661888c5aee77d/pydantic_core-2.46.4-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:5d5902252db0d3cedf8d4a1bc68f70eeb430f7e4c7104c8c476753519b423008", size = 2106306, upload-time = "2026-05-06T13:37:48.029Z" },
+    { url = "https://files.pythonhosted.org/packages/c1/81/4fa520eaffa8bd7d1525e644cd6d39e7d60b1592bc5b516693c7340b50f1/pydantic_core-2.46.4-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:c94f0688e7b8d0a67abf40e57a7eaaecd17cc9586706a31b76c031f63df052b4", size = 1951906, upload-time = "2026-05-06T13:37:17.012Z" },
+    { url = "https://files.pythonhosted.org/packages/03/d5/fd02da45b659668b05923b17ba3a0100a0a3d5541e3bd8fcc4ecb711309e/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f027324c56cd5406ca49c124b0db10e56c69064fec039acc571c29020cc87c76", size = 1976802, upload-time = "2026-05-06T13:37:35.113Z" },
+    { url = "https://files.pythonhosted.org/packages/21/f2/95727e1368be3d3ed485eaab7adbd7dda408f33f7a36e8b48e0144002b91/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:e739fee756ba1010f8bcccb534252e85a35fe45ae92c295a06059ce58b74ccd3", size = 2052446, upload-time = "2026-05-06T13:37:12.313Z" },
+    { url = "https://files.pythonhosted.org/packages/9c/86/5d99feea3f77c7234b8718075b23db11532773c1a0dbd9b9490215dc2eeb/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9d56801be94b86a9da183e5f3766e6310752b99ff647e38b09a9500d88e46e76", size = 2232757, upload-time = "2026-05-06T13:39:01.149Z" },
+    { url = "https://files.pythonhosted.org/packages/d2/3a/508ac615935ef7588cf6d9e9b91309fdc2da751af865e02a9098de88258c/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:2412e734dcb48da14d4e4006b82b46b74f2518b8a26ee7e58c6844a6cd6d03c4", size = 2309275, upload-time = "2026-05-06T13:37:41.406Z" },
+    { url = "https://files.pythonhosted.org/packages/07/f8/41db9de19d7987d6b04715a02b3b40aea467000275d9d758ffaa31af7d50/pydantic_core-2.46.4-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9551187363ffc0de2a00b2e47c25aeaeb1020b69b668762966df15fc5659dd5a", size = 2094467, upload-time = "2026-05-06T13:39:18.847Z" },
+    { url = "https://files.pythonhosted.org/packages/2c/e2/f35033184cb11d0052daf4416e8e10a502ea2ac006fc4f459aee872727d1/pydantic_core-2.46.4-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:0186750b482eefa11d7f435892b09c5c606193ef3375bcf94aa00ae6bfb66262", size = 2134417, upload-time = "2026-05-06T13:40:17.944Z" },
+    { url = "https://files.pythonhosted.org/packages/7e/7b/6ceeb1cc90e193862f444ebe373d8fdf613f0a82572dde03fb10734c6c71/pydantic_core-2.46.4-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:5855698a4856556d86e8e6cd8434bc3ac0314ee8e12089ae0e143f64c6256e4e", size = 2179782, upload-time = "2026-05-06T13:40:32.618Z" },
+    { url = "https://files.pythonhosted.org/packages/5a/f2/c8d7773ede6af08036423a00ae0ceffce266c3c52a096c435d68c896083f/pydantic_core-2.46.4-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:cbaf13819775b7f769bf4a1f066cb6df7a28d4480081a589828ef190226881cd", size = 2188782, upload-time = "2026-05-06T13:36:51.018Z" },
+    { url = "https://files.pythonhosted.org/packages/59/31/0c864784e31f09f05cdd87606f08923b9c9e7f6e51dd27f20f62f975ce9f/pydantic_core-2.46.4-cp313-cp313-musllinux_1_1_armv7l.whl", hash = "sha256:633147d34cf4550417f12e2b1a0383973bdf5cdfde212cb09e9a581cf10820be", size = 2328334, upload-time = "2026-05-06T13:40:37.764Z" },
+    { url = "https://files.pythonhosted.org/packages/c2/eb/4f6c8a41efa30baa755590f4141abf3a8c370fab610915733e74134a7270/pydantic_core-2.46.4-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:82cf5301172168103724d49a1444d3378cb20cdee30b116a1bd6031236298a5d", size = 2372986, upload-time = "2026-05-06T13:39:34.152Z" },
+    { url = "https://files.pythonhosted.org/packages/5b/24/b375a480d53113860c299764bfe9f349a3dc9108b3adc0d7f0d786492ebf/pydantic_core-2.46.4-cp313-cp313-win32.whl", hash = "sha256:9fa8ae11da9e2b3126c6426f147e0fba88d96d65921799bb30c6abd1cb2c97fb", size = 1973693, upload-time = "2026-05-06T13:37:55.072Z" },
+    { url = "https://files.pythonhosted.org/packages/7e/e8/cff247591966f2d22ec8c003cd7587e27b7ba7b81ab2fb888e3ab75dc285/pydantic_core-2.46.4-cp313-cp313-win_amd64.whl", hash = "sha256:6b3ace8194b0e5204818c92802dcdca7fc6d88aabbb799d7c795540d9cd6d292", size = 2071819, upload-time = "2026-05-06T13:38:49.139Z" },
+    { url = "https://files.pythonhosted.org/packages/c6/1a/f4aee670d5670e9e148e0c82c7db98d780be566c6e6a97ee8035528ca0b3/pydantic_core-2.46.4-cp313-cp313-win_arm64.whl", hash = "sha256:184c081504d17f1c1066e430e117142b2c77d9448a97f7b65c6ac9fd9aee238d", size = 2027411, upload-time = "2026-05-06T13:40:45.796Z" },
+    { url = "https://files.pythonhosted.org/packages/8d/74/228a26ddad29c6672b805d9fd78e8d251cd04004fa7eed0e622096cd0250/pydantic_core-2.46.4-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:428e04521a40150c85216fc8b85e8d39fece235a9cf5e383761238c7fa9b96fb", size = 2102079, upload-time = "2026-05-06T13:38:41.019Z" },
+    { url = "https://files.pythonhosted.org/packages/ad/1f/8970b150a4b4365623ae00fc88603491f763c627311ae8031e3111356d6e/pydantic_core-2.46.4-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:23ace664830ee0bfe014a0c7bc248b1f7f25ed7ad103852c317624a1083af462", size = 1952179, upload-time = "2026-05-06T13:36:59.812Z" },
+    { url = "https://files.pythonhosted.org/packages/95/30/5211a831ae054928054b2f79731661087a2bc5c01e825c672b3a4a8f1b3e/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ce5c1d2a8b27468f433ca974829c44060b8097eedc39933e3c206a90ee49c4a9", size = 1978926, upload-time = "2026-05-06T13:37:39.933Z" },
+    { url = "https://files.pythonhosted.org/packages/57/e9/689668733b1eb67adeef047db3c2e8788fcf65a7fd9c9e2b46b7744fe245/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:7283d57845ecf5a163403eb0702dfc220cc4fbdd18919cb5ccea4f95ee1cdab4", size = 2046785, upload-time = "2026-05-06T13:38:01.995Z" },
+    { url = "https://files.pythonhosted.org/packages/60/d9/6715260422ff50a2109878fd24d948a6c3446bb2664f34ee78cd972b3acd/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:8daafc69c93ee8a0204506a3b6b30f586ef54028f52aeeeb5c4cfc5184fd5914", size = 2228733, upload-time = "2026-05-06T13:40:50.371Z" },
+    { url = "https://files.pythonhosted.org/packages/18/ae/fdb2f64316afca925640f8e70bb1a564b0ec2721c1389e25b8eb4bf9a299/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:cd2213145bcc2ba85884d0ac63d222fece9209678f77b9b4d76f054c561adb28", size = 2307534, upload-time = "2026-05-06T13:37:21.531Z" },
+    { url = "https://files.pythonhosted.org/packages/89/1d/8eff589b45bb8190a9d12c49cfad0f176a5cbd1534908a6b5125e2886239/pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7a5f930472650a82629163023e630d160863fce524c616f4e5186e5de9d9a49b", size = 2099732, upload-time = "2026-05-06T13:39:31.942Z" },
+    { url = "https://files.pythonhosted.org/packages/06/d5/ee5a3366637fee41dee51a1fc91562dcf12ddbc68fda34e6b253da2324bb/pydantic_core-2.46.4-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:c1b3f518abeca3aa13c712fd202306e145abf59a18b094a6bafb2d2bbf59192c", size = 2129627, upload-time = "2026-05-06T13:37:25.033Z" },
+    { url = "https://files.pythonhosted.org/packages/94/33/2414be571d2c6a6c4d08be21f9292b6d3fdb08949a97b6dfe985017821db/pydantic_core-2.46.4-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:1a7dd0b3ee80d90150e3495a3a13ac34dbcbfd4f012996a6a1d8900e91b5c0fb", size = 2179141, upload-time = "2026-05-06T13:37:14.046Z" },
+    { url = "https://files.pythonhosted.org/packages/7b/79/7daa95be995be0eecc4cf75064cb33f9bbbfe3fe0158caf2f0d4a996a5c7/pydantic_core-2.46.4-cp314-cp314-musllinux_1_1_aarch64.whl", hash = "sha256:3fb702cd90b0446a3a1c5e470bfa0dd23c0233b676a9099ddcc964fa6ca13898", size = 2184325, upload-time = "2026-05-06T13:36:53.615Z" },
+    { url = "https://files.pythonhosted.org/packages/9f/cb/d0a382f5c0de8a222dc61c65348e0ce831b1f68e0a018450d31c2cace3a5/pydantic_core-2.46.4-cp314-cp314-musllinux_1_1_armv7l.whl", hash = "sha256:b8458003118a712e66286df6a707db01c52c0f52f7db8e4a38f0da1d3b94fc4e", size = 2323990, upload-time = "2026-05-06T13:40:29.971Z" },
+    { url = "https://files.pythonhosted.org/packages/05/db/d9ba624cc4a5aced1598e88c04fdbd8310c8a69b9d38b9a3d39ce3a61ed7/pydantic_core-2.46.4-cp314-cp314-musllinux_1_1_x86_64.whl", hash = "sha256:372429a130e469c9cd698925ce5fc50940b7a1336b0d82038e63d5bbc4edc519", size = 2369978, upload-time = "2026-05-06T13:37:23.027Z" },
+    { url = "https://files.pythonhosted.org/packages/f2/20/d15df15ba918c423461905802bfd2981c3af0bfa0e40d05e13edbfa48bc3/pydantic_core-2.46.4-cp314-cp314-win32.whl", hash = "sha256:85bb3611ff1802f3ee7fdd7dbff26b56f343fb432d57a4728fdd49b6ef35e2f4", size = 1966354, upload-time = "2026-05-06T13:38:03.499Z" },
+    { url = "https://files.pythonhosted.org/packages/fc/b6/6b8de4c0a7d7ab3004c439c80c5c1e0a3e8d78bbae19379b01960383d9e5/pydantic_core-2.46.4-cp314-cp314-win_amd64.whl", hash = "sha256:811ff8e9c313ab425368bcbb36e5c4ebd7108c2bbf4e4089cfbb0b01eff63fac", size = 2072238, upload-time = "2026-05-06T13:39:40.807Z" },
+    { url = "https://files.pythonhosted.org/packages/32/36/51eb763beec1f4cf59b1db243a7dcc39cbb41230f050a09b9d69faaf0a48/pydantic_core-2.46.4-cp314-cp314-win_arm64.whl", hash = "sha256:bfec22eab3c8cc2ceec0248aec886624116dc079afa027ecc8ad4a7e62010f8a", size = 2018251, upload-time = "2026-05-06T13:37:26.72Z" },
+    { url = "https://files.pythonhosted.org/packages/e8/91/855af51d625b23aa987116a19e231d2aaef9c4a415273ddc189b79a45fee/pydantic_core-2.46.4-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:af8244b2bef6aaad6d92cda81372de7f8c8d36c9f0c3ea36e827c60e7d9467a0", size = 2099593, upload-time = "2026-05-06T13:39:47.682Z" },
+    { url = "https://files.pythonhosted.org/packages/fb/1b/8784a54c65edb5f49f0a14d6977cf1b209bba85a4c77445b255c2de58ab3/pydantic_core-2.46.4-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:5a4330cdbc57162e4b3aa303f588ba752257694c9c9be3e7ebb11b4aca659b5d", size = 1935226, upload-time = "2026-05-06T13:40:40.428Z" },
+    { url = "https://files.pythonhosted.org/packages/e8/e7/1955d28d1afc56dd4b3ad7cc0cf39df1b9852964cf16e5d13912756d6d6b/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:29c61fc04a3d840155ff08e475a04809278972fe6aef51e2720554e96367e34b", size = 1974605, upload-time = "2026-05-06T13:37:32.029Z" },
+    { url = "https://files.pythonhosted.org/packages/93/e2/3fedbf0ba7a22850e6e9fd78117f1c0f10f950182344d8a6c535d468fdd8/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:c50f2528cf200c5eed56faf3f4e22fcd5f38c157a8b78576e6ba3168ec35f000", size = 2030777, upload-time = "2026-05-06T13:38:55.239Z" },
+    { url = "https://files.pythonhosted.org/packages/f8/61/46be275fcaaba0b4f5b9669dd852267ce1ff616592dccf7a7845588df091/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:0cbe8b01f948de4286c74cdd6c667aceb38f5c1e26f0693b3983d9d74887c65e", size = 2236641, upload-time = "2026-05-06T13:37:08.096Z" },
+    { url = "https://files.pythonhosted.org/packages/60/db/12e93e46a8bac9988be3c016860f83293daea8c716c029c9ace279036f2f/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:617d7e2ca7dcb8c5cf6bcb8c59b8832c94b36196bbf1cbd1bfb56ed341905edd", size = 2286404, upload-time = "2026-05-06T13:40:20.221Z" },
+    { url = "https://files.pythonhosted.org/packages/e2/4a/4d8b19008f38d31c53b8219cfedc2e3d5de5fe99d90076b7e767de29274f/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7027560ee92211647d0d34e3f7cd6f50da56399d26a9c8ad0da286d3869a53f3", size = 2109219, upload-time = "2026-05-06T13:38:12.153Z" },
+    { url = "https://files.pythonhosted.org/packages/88/70/3cbc40978fefb7bb09c6708d40d4ad1a5d70fd7213c3d17f971de868ec1f/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:f99626688942fb746e545232e7726926f3be91b5975f8b55327665fafda991c7", size = 2110594, upload-time = "2026-05-06T13:40:02.971Z" },
+    { url = "https://files.pythonhosted.org/packages/9d/20/b8d36736216e29491125531685b2f9e61aa5b4b2599893f8268551da3338/pydantic_core-2.46.4-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:fc3e9034a63de20e15e8ade85358bc6efc614008cab72898b4b4952bea0509ff", size = 2159542, upload-time = "2026-05-06T13:39:27.506Z" },
+    { url = "https://files.pythonhosted.org/packages/1d/a2/367df868eb584dacf6bf82a389272406d7178e301c4ac82545ab98bc2dd9/pydantic_core-2.46.4-cp314-cp314t-musllinux_1_1_aarch64.whl", hash = "sha256:97e7cf2be5c77b7d1a9713a05605d49460d02c6078d38d8bef3cbe323c548424", size = 2168146, upload-time = "2026-05-06T13:38:31.93Z" },
+    { url = "https://files.pythonhosted.org/packages/c1/b8/4460f77f7e201893f649a29ab355dddd3beee8a97bcb1a320db414f9a06e/pydantic_core-2.46.4-cp314-cp314t-musllinux_1_1_armv7l.whl", hash = "sha256:3bf92c5d0e00fefaab325a4d27828fe6b6e2a21848686b5b60d2d9eeb09d76c6", size = 2306309, upload-time = "2026-05-06T13:37:44.717Z" },
+    { url = "https://files.pythonhosted.org/packages/64/c4/be2639293acd87dc8ddbcec41a73cee9b2ebf996fe6d892a1a74e88ad3f7/pydantic_core-2.46.4-cp314-cp314t-musllinux_1_1_x86_64.whl", hash = "sha256:3ecbc122d18468d06ca279dc26a8c2e2d5acb10943bb35e36ae92096dc3b5565", size = 2369736, upload-time = "2026-05-06T13:37:05.645Z" },
+    { url = "https://files.pythonhosted.org/packages/30/a6/9f9f380dbb301f67023bf8f707aaa75daadf84f7152d95c410fd7e81d994/pydantic_core-2.46.4-cp314-cp314t-win32.whl", hash = "sha256:e846ae7835bf0703ae43f534ab79a867146dadd59dc9ca5c8b53d5c8f7c9ef02", size = 1955575, upload-time = "2026-05-06T13:38:51.116Z" },
+    { url = "https://files.pythonhosted.org/packages/40/1f/f1eb9eb350e795d1af8586289746f5c5677d16043040d63710e22abc43c9/pydantic_core-2.46.4-cp314-cp314t-win_amd64.whl", hash = "sha256:2108ba5c1c1eca18030634489dc544844144ee36357f2f9f780b93e7ddbb44b5", size = 2051624, upload-time = "2026-05-06T13:38:21.672Z" },
+    { url = "https://files.pythonhosted.org/packages/f6/d2/42dd53d0a85c27606f316d3aa5d2869c4e8470a5ed6dec30e4a1abe19192/pydantic_core-2.46.4-cp314-cp314t-win_arm64.whl", hash = "sha256:4fcbe087dbc2068af7eda3aa87634eba216dbda64d1ae73c8684b621d33f6596", size = 2017325, upload-time = "2026-05-06T13:40:52.723Z" },
+    { url = "https://files.pythonhosted.org/packages/ee/a4/73995fd4ebbb46ba0ee51e6fa049b8f02c40daebb762208feda8a6b7894d/pydantic_core-2.46.4-graalpy311-graalpy242_311_native-macosx_10_12_x86_64.whl", hash = "sha256:14d4edf427bdcf950a8a02d7cb44a08614388dd6e1bdcbf4f67504fa7887da9c", size = 2111589, upload-time = "2026-05-06T13:37:10.817Z" },
+    { url = "https://files.pythonhosted.org/packages/fb/7f/f37d3a5e8bfcc2e403f5c57a730f2d815693fb42119e8ea48b3789335af1/pydantic_core-2.46.4-graalpy311-graalpy242_311_native-macosx_11_0_arm64.whl", hash = "sha256:0ce40cd7b21210e99342afafbd4d0f76d784eb5b1d60f3bdc566be4983c6c73b", size = 1944552, upload-time = "2026-05-06T13:36:56.717Z" },
+    { url = "https://files.pythonhosted.org/packages/15/3c/d7eb777b3ff43e8433a4efb39a17aa8fd98a4ee8561a24a67ef5db07b2d6/pydantic_core-2.46.4-graalpy311-graalpy242_311_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:90884113d8b48f760e9587002789ddd741e76ab9f89518cd1e43b1f1a52ec44b", size = 1982984, upload-time = "2026-05-06T13:39:06.207Z" },
+    { url = "https://files.pythonhosted.org/packages/63/87/70b9f40170a81afd55ca26c9b2acb25c20d64bcfbf888fafecb3ba077d4c/pydantic_core-2.46.4-graalpy311-graalpy242_311_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:66ce7632c22d837c95301830e111ad0128a32b8207533b60896a96c4915192ea", size = 2138417, upload-time = "2026-05-06T13:39:45.476Z" },
+    { url = "https://files.pythonhosted.org/packages/9d/1d/8987ad40f65ae1432753072f214fb5c74fe47ffbd0698bb9cbbb585664f8/pydantic_core-2.46.4-graalpy312-graalpy250_312_native-macosx_10_12_x86_64.whl", hash = "sha256:1d8ba486450b14f3b1d63bc521d410ec7565e52f887b9fb671791886436a42f7", size = 2095527, upload-time = "2026-05-06T13:39:52.283Z" },
+    { url = "https://files.pythonhosted.org/packages/64/d3/84c282a7eee1d3ac4c0377546ef5a1ea436ce26840d9ac3b7ed54a377507/pydantic_core-2.46.4-graalpy312-graalpy250_312_native-macosx_11_0_arm64.whl", hash = "sha256:3009f12e4e90b7f88b4f9adb1b0c4a3d58fe7820f3238c190047209d148026df", size = 1936024, upload-time = "2026-05-06T13:40:15.671Z" },
+    { url = "https://files.pythonhosted.org/packages/d7/ca/eac61596cdeb4d7e174d3dc0bd8a6238f14f75f97a24e7b7db4c7e7340a0/pydantic_core-2.46.4-graalpy312-graalpy250_312_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ad785e92e6dc634c21555edc8bd6b64957ab844541bcb96a1366c202951ae526", size = 1990696, upload-time = "2026-05-06T13:38:34.717Z" },
+    { url = "https://files.pythonhosted.org/packages/fa/c3/7c8b240552251faf6b3a957db200fcfbbcec36763c050428b601e0c9b83b/pydantic_core-2.46.4-graalpy312-graalpy250_312_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:00c603d540afdd6b80eb39f078f33ebd46211f02f33e34a32d9f053bba711de0", size = 2147590, upload-time = "2026-05-06T13:39:29.883Z" },
+    { url = "https://files.pythonhosted.org/packages/11/cb/428de0385b6c8d44b716feba566abfacfbd23ee3c4439faa789a1456242f/pydantic_core-2.46.4-pp311-pypy311_pp73-macosx_10_12_x86_64.whl", hash = "sha256:0c563b08bca408dc7f65f700633d8442fffb2421fc47b8101377e9fd65051ff0", size = 2112782, upload-time = "2026-05-06T13:37:04.016Z" },
+    { url = "https://files.pythonhosted.org/packages/0b/b5/6a17bdadd0fc1f170adfd05a20d37c832f52b117b4d9131da1f41bb097ce/pydantic_core-2.46.4-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:db06ffe51636ffe9ca531fe9023dd64bdd794be8754cb5df57c5498ae5b518a7", size = 1952146, upload-time = "2026-05-06T13:39:43.092Z" },
+    { url = "https://files.pythonhosted.org/packages/2a/dc/03734d80e362cd43ef65428e9de77c730ce7f2f11c60d2b1e1b39f0fbf99/pydantic_core-2.46.4-pp311-pypy311_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:133878133d271ade3d41d1bfb2a45ec38dbdbda40bc065921c6b04e4630127e2", size = 2134492, upload-time = "2026-05-06T13:36:58.124Z" },
+    { url = "https://files.pythonhosted.org/packages/de/df/5e5ffc085ed07cc22d298134d3d911c63e91f6a0eb91fe646750a3209910/pydantic_core-2.46.4-pp311-pypy311_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:9bc519fbf2b7578398853d815009ae5e4d4603d12f4e3f91da8c06852d3da3e9", size = 2156604, upload-time = "2026-05-06T13:37:49.88Z" },
+    { url = "https://files.pythonhosted.org/packages/81/44/6e112a4253e56f5705467cbab7ab5e91ee7398ba3d56d358635958893d3e/pydantic_core-2.46.4-pp311-pypy311_pp73-musllinux_1_1_aarch64.whl", hash = "sha256:c7a7bd4e39e8e4c12c39cd480356842b6a8a06e41b23a55a5e3e191718838ddf", size = 2183828, upload-time = "2026-05-06T13:37:43.053Z" },
+    { url = "https://files.pythonhosted.org/packages/ac/ad/5565071e937d8e752842ac241463944c9eb14c87e2d269f2658a5bd05e98/pydantic_core-2.46.4-pp311-pypy311_pp73-musllinux_1_1_armv7l.whl", hash = "sha256:d396ec2b979760aaf3218e76c24e65bd0aca24983298653b3a9d7a45f9e47b30", size = 2310000, upload-time = "2026-05-06T13:37:56.694Z" },
+    { url = "https://files.pythonhosted.org/packages/4f/c3/66883a5cec183e7fba4d024b4cbbe61851a63750ef606b0afecc46d1f2bf/pydantic_core-2.46.4-pp311-pypy311_pp73-musllinux_1_1_x86_64.whl", hash = "sha256:86e1a4418c6cd97d60c95c71164158eaf7324fae7b0923264016baa993eba6fc", size = 2361286, upload-time = "2026-05-06T13:40:05.667Z" },
+    { url = "https://files.pythonhosted.org/packages/4b/2d/69abac8f838090bbecd5df894befb2c2619e7996a98ddb949db9f3b93225/pydantic_core-2.46.4-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:d51026d73fcfd93610abc7b27789c26b313920fcfb20e27462d74a7f8b06e983", size = 2193071, upload-time = "2026-05-06T13:38:08.682Z" },
+]
+
 [[package]]
 name = "pygments"
 version = "2.20.0"
@@ -320,6 +501,19 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl", hash = "sha256:2c5efc453d45394fdd706ade797c0a81091eccd1d6e4bccfcd476e2b8e0ab5d9", size = 375249, upload-time = "2026-04-07T17:16:16.13Z" },
 ]
 
+[[package]]
+name = "starlette"
+version = "1.1.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "anyio" },
+    { name = "typing-extensions", marker = "python_full_version < '3.13'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/95/66/4d20cdf39a8d6a51e663b7038e3b828ff211d3891a43a713fe7e4643f3a8/starlette-1.1.0.tar.gz", hash = "sha256:e83c7fe0ddecd8719c5b840080325aec0260acec86e9832899e377b91d65e90f", size = 2660060, upload-time = "2026-05-23T16:55:41.376Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/93/79/920b8e0a8b20f793e8d64855095cb8febabf6175b8550b6f7a547d813891/starlette-1.1.0-py3-none-any.whl", hash = "sha256:7f0dfd38e428aad5cb6f9f667f0ca1d2d8ca3f3385dccac8305f79ec98458382", size = 72899, upload-time = "2026-05-23T16:55:39.201Z" },
+]
+
 [[package]]
 name = "tomli"
 version = "2.4.1"
@@ -382,3 +576,29 @@ sdist = { url = "https://files.pythonhosted.org/packages/72/94/1a15dd82efb362ac8
 wheels = [
     { url = "https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548", size = 44614, upload-time = "2025-08-25T13:49:24.86Z" },
 ]
+
+[[package]]
+name = "typing-inspection"
+version = "0.4.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/55/e3/70399cb7dd41c10ac53367ae42139cf4b1ca5f36bb3dc6c9d33acdb43655/typing_inspection-0.4.2.tar.gz", hash = "sha256:ba561c48a67c5958007083d386c3295464928b01faa735ab8547c5692e87f464", size = 75949, upload-time = "2025-10-01T02:14:41.687Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/dc/9b/47798a6c91d8bdb567fe2698fe81e0c6b7cb7ef4d13da4114b41d239f65d/typing_inspection-0.4.2-py3-none-any.whl", hash = "sha256:4ed1cacbdc298c220f1bd249ed5287caa16f34d44ef4e9c3d0cbad5b521545e7", size = 14611, upload-time = "2025-10-01T02:14:40.154Z" },
+]
+
+[[package]]
+name = "uvicorn"
+version = "0.48.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "click" },
+    { name = "h11" },
+    { name = "typing-extensions", marker = "python_full_version < '3.11'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/e6/bf/f6544ba992ddb9a6077343a576f9844f7f8f06ab819aefd00206e9255f18/uvicorn-0.48.0.tar.gz", hash = "sha256:a5504207195d08c2511bf9125ede5ac4a4b71725d519e758d01dcf0bc2d31c37", size = 91074, upload-time = "2026-05-24T12:08:41.925Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/01/be/72532be3da7acc5fdfbccdb95215cd04f995a0886532a5b423f929cda4cc/uvicorn-0.48.0-py3-none-any.whl", hash = "sha256:48097851328b87ec36117d3d575234519eb58c2b22d79666e9bbc6c49a761dad", size = 71410, upload-time = "2026-05-24T12:08:40.258Z" },
+]