Connector tax Phase A: build_connector_meta is 1.4ms/step (the tax source)
Per-step timing from engine_step.jsonl definitively resolves H3: plain: 53 μs/step (p50) noop_connector: 69 μs/step (+16 μs = negligible framework cost) mooncake_producer: 1461 μs/step (build_connector_meta = 1386 μs) mooncake_both: 1452 μs/step (same as producer) The substrate tax is NOT in the v1 framework — it's specifically in Mooncake's build_connector_meta() which walks set(cache.keys()) every scheduler step (O(|cache|) per step, E2 audit §6.5). Accumulated per-request tax: 256 decode steps × 1.4ms = 358ms. Observed TTFT tax at rate=1.0: plain 378ms vs mooncake_both 422ms (+12%). At rate=2.0 (near saturation): +29%, approaching trace-replay's +45%. Also fixes kill_vllm() to properly kill EngineCore subprocesses.
This commit is contained in:
@@ -58,22 +58,29 @@ manifest() {
|
||||
|
||||
kill_vllm() {
|
||||
local pidfile="$1"
|
||||
# Kill the API server PID
|
||||
if [[ -f "$pidfile" ]]; then
|
||||
local pid; pid=$(cat "$pidfile")
|
||||
if [[ -n "$pid" ]]; then
|
||||
kill -9 "$pid" 2>/dev/null || true
|
||||
fi
|
||||
fi
|
||||
pkill -f "port $PORT" 2>/dev/null || true
|
||||
# Also kill any vLLM/EngineCore processes on this port or GPU
|
||||
pkill -9 -f "port $PORT" 2>/dev/null || true
|
||||
pkill -9 -f "VLLM::EngineCore" 2>/dev/null || true
|
||||
pkill -9 -f "vllm.entrypoints" 2>/dev/null || true
|
||||
sleep 5
|
||||
# wait for GPU release
|
||||
# Wait for GPU memory release (up to 60s)
|
||||
for _ in $(seq 1 30); do
|
||||
used=$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits | head -n $((GPU_ID + 1)) | tail -1)
|
||||
if [[ "$used" -lt 1000 ]]; then
|
||||
break
|
||||
used=$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i "$GPU_ID" 2>/dev/null | tr -d ' ')
|
||||
if [[ -n "$used" && "$used" -lt 1000 ]]; then
|
||||
return 0
|
||||
fi
|
||||
# Try killing any remaining GPU holders
|
||||
fuser -k "/dev/nvidia${GPU_ID}" 2>/dev/null || true
|
||||
sleep 2
|
||||
done
|
||||
echo "WARNING: GPU $GPU_ID still not free after 60s" >&2
|
||||
}
|
||||
|
||||
run_phase_a() {
|
||||
|
||||
Reference in New Issue
Block a user