A3: vLLM scheduler patch for step-level JSONL log

When AGENTIC_STEP_LOG_PATH is set, the scheduler emits one JSONL line
per scheduler step with t_unix, worker_id, prefill/decode token
counts, n_running/n_waiting, preempted ids, and per-request phase
labels. No-op when the env var is unset, so production engines are
not impacted. bench.sh now threads AGENTIC_STEP_LOG_DIR through to
each per-engine launch so step logs end up at engine_${i}.jsonl.

Required by Batch 2 (PD-colo interference index) and Batch 5
(same-worker overlap attribution); engine /metrics polling cannot
provide per-step granularity.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-05-25 16:19:11 +08:00
parent fe556b5d98
commit 5816aad731
3 changed files with 175 additions and 0 deletions

View File

@@ -161,12 +161,24 @@ launch_instances() {
use_mooncake=true
fi
# Optional: when AGENTIC_STEP_LOG_DIR is exported, point each engine at its
# own JSONL file so the patched scheduler emits per-step records.
local step_log_dir="${AGENTIC_STEP_LOG_DIR:-}"
if [ -n "$step_log_dir" ]; then
mkdir -p "$step_log_dir"
fi
for i in $(seq 0 $((N_INSTANCES - 1))); do
local port=$((BASE_PORT + i))
local master=$((29500 + i))
local logfile="$OUTDIR/vllm_inst_${i}.log"
local step_env=""
if [ -n "$step_log_dir" ]; then
step_env="AGENTIC_STEP_LOG_PATH=$step_log_dir/engine_${i}.jsonl AGENTIC_WORKER_ID=engine_${i}"
fi
if [ "$use_mooncake" = "true" ]; then
env $step_env \
PYTHONHASHSEED=42 \
VLLM_MOONCAKE_BOOTSTRAP_PORT=$((8998 + i)) \
MASTER_PORT=$master \
@@ -180,6 +192,7 @@ launch_instances() {
$vllm_extra_args \
> "$logfile" 2>&1 &
else
env $step_env \
MASTER_PORT=$master \
CUDA_VISIBLE_DEVICES=$i \
$VLLM serve "$MODEL" \