A3: vLLM scheduler patch for step-level JSONL log
When AGENTIC_STEP_LOG_PATH is set, the scheduler emits one JSONL line
per scheduler step with t_unix, worker_id, prefill/decode token
counts, n_running/n_waiting, preempted ids, and per-request phase
labels. No-op when the env var is unset, so production engines are
not impacted. bench.sh now threads AGENTIC_STEP_LOG_DIR through to
each per-engine launch so step logs end up at engine_${i}.jsonl.
Required by Batch 2 (PD-colo interference index) and Batch 5
(same-worker overlap attribution); engine /metrics polling cannot
provide per-step granularity.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -161,12 +161,24 @@ launch_instances() {
|
||||
use_mooncake=true
|
||||
fi
|
||||
|
||||
# Optional: when AGENTIC_STEP_LOG_DIR is exported, point each engine at its
|
||||
# own JSONL file so the patched scheduler emits per-step records.
|
||||
local step_log_dir="${AGENTIC_STEP_LOG_DIR:-}"
|
||||
if [ -n "$step_log_dir" ]; then
|
||||
mkdir -p "$step_log_dir"
|
||||
fi
|
||||
|
||||
for i in $(seq 0 $((N_INSTANCES - 1))); do
|
||||
local port=$((BASE_PORT + i))
|
||||
local master=$((29500 + i))
|
||||
local logfile="$OUTDIR/vllm_inst_${i}.log"
|
||||
local step_env=""
|
||||
if [ -n "$step_log_dir" ]; then
|
||||
step_env="AGENTIC_STEP_LOG_PATH=$step_log_dir/engine_${i}.jsonl AGENTIC_WORKER_ID=engine_${i}"
|
||||
fi
|
||||
|
||||
if [ "$use_mooncake" = "true" ]; then
|
||||
env $step_env \
|
||||
PYTHONHASHSEED=42 \
|
||||
VLLM_MOONCAKE_BOOTSTRAP_PORT=$((8998 + i)) \
|
||||
MASTER_PORT=$master \
|
||||
@@ -180,6 +192,7 @@ launch_instances() {
|
||||
$vllm_extra_args \
|
||||
> "$logfile" 2>&1 &
|
||||
else
|
||||
env $step_env \
|
||||
MASTER_PORT=$master \
|
||||
CUDA_VISIBLE_DEVICES=$i \
|
||||
$VLLM serve "$MODEL" \
|
||||
|
||||
Reference in New Issue
Block a user