MB2 scaffolding: launch script for vLLM pair + KV-transfer-time client

Two new files prepare measurement of T_transfer(KV_size, network_path),
the gap §3.2's PD-disagg cost argument has had since day one.

microbench/fresh_setup/start_vllm_pair.sh
  start | status | stop two vLLM 0.18.1 instances on local GPUs (A, B)
  with --kv-transfer-config '{"kv_connector":"MooncakeConnector",
  "kv_role":"kv_both"}' running off the fresh venv (vanilla wheel +
  vanilla mooncake 0.3.11, NOT the dash0 patched build). GPU IDs and
  ports are env-overridable so the same script drives the intra-node
  pair (GPU_A=0 GPU_B=1 on one host) and the inter-node pair (GPU_A=0
  on dash1, GPU_B=0 on dash2 — launched per host separately).

microbench/fresh_setup/mb2_kv_transfer.py
  Three-step measurement borrowed from connector_tax/.../smoke_test_
  migrate_cache.py:
    1. do_remote_decode  on A   (compute & cache KV; max_tokens=1)
    2. do_remote_prefill on B   (pull KV from A — this is the timed step)
    3. plain completion on B    (sanity check: cached_tokens ≈ prompt len)
  Sweeps input_tokens ∈ {512, 1k, 2k, 4k, 8k, 16k, 32k, 64k} with 5
  repeats each; reports mean / p50 / p90 transfer time and a per-size
  raw log. Per-token KV is 98304 B (Qwen3-Coder-30B-A3B), so the upper
  end ≈ 6 GiB transfers — within the p99 11.5 GiB range from §2 but
  below it (the model's max_model_len 200000 caps the absolute upper).

What we will NOT learn from this design:
  - Bandwidth saturation when the system is loaded (single-request bench)
  - vLLM-internal scheduling overhead vs pure transfer (the timed step
    folds them together — but for the §3.2 argument that's the right
    "what does PD-disagg actually pay" number)

Intentionally not committed yet: an orchestrator that loops over
intra-/inter-node configs. We start manual on dash1 intra-node to
verify the measurement is sane before scaling out.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

microbench/fresh_setup/start_vllm_pair.sh

@@ -0,0 +1,105 @@
+#!/usr/bin/env bash
+# Start 2 vLLM instances with Mooncake kv_connector (kv_both) for MB2.
+#
+# Default config: both on local GPU 0 and 1 (intra-node A/B test).
+# Override via GPU_A / GPU_B / HOST_A / HOST_B env vars.
+#
+# This uses the FRESH venv at /home/admin/cpfs/wjh/agentic-kv-fresh/.venv
+# (vanilla vllm 0.18.1 + vanilla mooncake-transfer-engine 0.3.11), NOT
+# the dash0 patched build.
+#
+# Usage:
+#   GPU_A=0 GPU_B=1 bash microbench/fresh_setup/start_vllm_pair.sh
+#   bash microbench/fresh_setup/start_vllm_pair.sh status
+#   bash microbench/fresh_setup/start_vllm_pair.sh stop
+
+set -eo pipefail
+
+FRESH_ROOT="/home/admin/cpfs/wjh/agentic-kv-fresh"
+VENV="${FRESH_ROOT}/.venv"
+MODEL="${MODEL:-/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct}"
+LOGS_DIR="${LOGS_DIR:-${FRESH_ROOT}/mb2_logs}"
+mkdir -p "${LOGS_DIR}"
+
+GPU_A="${GPU_A:-0}"
+GPU_B="${GPU_B:-1}"
+PORT_A=8000
+PORT_B=8001
+BP_A=8998
+BP_B=8999
+MASTER_A=29500
+MASTER_B=29501
+
+stop_all() {
+    pkill -9 -f "vllm serve" 2>/dev/null || true
+    pkill -9 -f "EngineCore" 2>/dev/null || true
+    sleep 2
+}
+
+case "${1:-start}" in
+    stop)
+        stop_all
+        exit 0
+        ;;
+    status)
+        for p in "${PORT_A}" "${PORT_B}"; do
+            if curl -sf "http://127.0.0.1:${p}/health" >/dev/null 2>&1; then
+                echo "port ${p}: UP"
+            else
+                echo "port ${p}: DOWN"
+            fi
+        done
+        exit 0
+        ;;
+    start)
+        ;;
+    *)
+        echo "Unknown command: $1"; exit 1;;
+esac
+
+stop_all
+
+source "${VENV}/bin/activate"
+
+launch() {
+    local idx="$1" gpu="$2" port="$3" bp="$4" master="$5"
+    echo "[mb2] launching instance ${idx} on GPU ${gpu}, port ${port}, bp ${bp}"
+    PYTHONHASHSEED=42 \
+    VLLM_MOONCAKE_BOOTSTRAP_PORT="${bp}" \
+    CUDA_VISIBLE_DEVICES="${gpu}" \
+    MASTER_PORT="${master}" \
+    nohup vllm serve "${MODEL}" \
+        --host 0.0.0.0 --port "${port}" \
+        --tensor-parallel-size 1 \
+        --trust-remote-code --enable-prefix-caching \
+        --dtype auto --gpu-memory-utilization 0.9 \
+        --max-model-len 200000 \
+        --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_both"}' \
+        --enable-prompt-tokens-details \
+        > "${LOGS_DIR}/vllm_${idx}_gpu${gpu}.log" 2>&1 &
+    disown
+}
+
+launch A "${GPU_A}" "${PORT_A}" "${BP_A}" "${MASTER_A}"
+sleep 3
+launch B "${GPU_B}" "${PORT_B}" "${BP_B}" "${MASTER_B}"
+
+echo "[mb2] waiting for both /health endpoints..."
+for port in "${PORT_A}" "${PORT_B}"; do
+    tries=0
+    while ! curl -sf "http://127.0.0.1:${port}/health" >/dev/null 2>&1; do
+        tries=$((tries+1))
+        if [ ${tries} -gt 180 ]; then
+            echo "[mb2] FATAL port ${port} did not come up in 6 min"
+            tail -40 "${LOGS_DIR}/vllm_"*"_gpu"*".log" || true
+            exit 1
+        fi
+        sleep 2
+    done
+    echo "  port=${port} ready"
+done
+
+echo "[mb2] both instances UP"
+echo "  A: 127.0.0.1:${PORT_A} (GPU ${GPU_A}, bp ${BP_A})"
+echo "  B: 127.0.0.1:${PORT_B} (GPU ${GPU_B}, bp ${BP_B})"
+echo "  logs: ${LOGS_DIR}"