gahow/agentic-kvc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Paper section: PD-sep scaffold + drop --enforce-eager from launch scripts

Browse Source 
Adds analysis/pd_sep_paper_section/ as the home for the "PD separation is
net negative under agentic workloads" paper section: plot scripts for C1
(workload chars), C6 (roofline), C7 (routing-vs-PD-sep lever), the C6/C7
PDFs already rendered, and a README mapping candidate claims to required
figures plus open re-run items.

Removes --enforce-eager from bench.sh and all active launch scripts so
cuda graphs are captured -- the prior methodology suppressed one of
PD-sep's structural advantages (D-node fixed-shape decode). Legacy
scripts under scripts/legacy/ are intentionally untouched as historical
records.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Gahow Wang
2026-05-25 11:24:16 +08:00
parent 6a27f75337
commit d71a111099
11 changed files with 576 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
scripts/bench.sh
View File

@@ -147,7 +147,7 @@ launch_instances() {
$VLLM serve "$MODEL" \
--host 0.0.0.0 --port $port \
--tensor-parallel-size 1 \
--trust-remote-code --enable-prefix-caching --enforce-eager \
--trust-remote-code --enable-prefix-caching \
--dtype auto --gpu-memory-utilization 0.9 --max-model-len 200000 \
--kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_both"}' \
$vllm_extra_args \
@@ -158,7 +158,7 @@ launch_instances() {
$VLLM serve "$MODEL" \
--host 0.0.0.0 --port $port \
--tensor-parallel-size 1 \
--trust-remote-code --enable-prefix-caching --enforce-eager \
--trust-remote-code --enable-prefix-caching \
--dtype auto --gpu-memory-utilization 0.9 --max-model-len 200000 \
$vllm_extra_args \
> "$logfile" 2>&1 &

2
scripts/launch_elastic_p2p.sh
View File

@@ -54,7 +54,7 @@ for i in $(seq 0 $((N_INSTANCES - 1))); do
$VLLM serve "$MODEL" \
--host 0.0.0.0 --port $port \
--tensor-parallel-size 1 \
--trust-remote-code --enable-prefix-caching --enforce-eager \
--trust-remote-code --enable-prefix-caching \
--dtype auto --gpu-memory-utilization 0.9 --max-model-len 200000 \
--kv-transfer-config \
'{"kv_connector":"MooncakeConnector","kv_role":"kv_both"}' \

2
scripts/launch_pd_mooncake.sh
View File

@@ -44,7 +44,6 @@ $VLLM serve "$MODEL_PATH" \
--tensor-parallel-size 4 \
--trust-remote-code \
--enable-prefix-caching \
--enforce-eager \
--dtype auto \
--gpu-memory-utilization 0.9 \
--kv-transfer-config \
@@ -61,7 +60,6 @@ $VLLM serve "$MODEL_PATH" \
--tensor-parallel-size 4 \
--trust-remote-code \
--enable-prefix-caching \
--enforce-eager \
--dtype auto \
--gpu-memory-utilization 0.8 \
--kv-transfer-config \

2
scripts/launch_pd_separated.sh
View File

@@ -49,7 +49,6 @@ CUDA_VISIBLE_DEVICES=0,1,2,3 $VLLM serve "$MODEL_PATH" \
--tensor-parallel-size 4 \
--trust-remote-code \
--enable-prefix-caching \
--enforce-eager \
--dtype auto \
--gpu-memory-utilization 0.9 \
--kv-transfer-config \
@@ -65,7 +64,6 @@ CUDA_VISIBLE_DEVICES=4,5,6,7 $VLLM serve "$MODEL_PATH" \
--tensor-parallel-size 4 \
--trust-remote-code \
--enable-prefix-caching \
--enforce-eager \
--dtype auto \
--gpu-memory-utilization 0.8 \
--kv-transfer-config \

4
scripts/launch_phase1_ps.sh
View File

@@ -29,7 +29,7 @@ for i in $(seq 0 6); do
echo "Starting C instance $i on GPU $i, port $((8000+i)), bootstrap $((8998+i))"
VLLM_MOONCAKE_BOOTSTRAP_PORT=$((8998+i)) MASTER_PORT=$((29500+i)) CUDA_VISIBLE_DEVICES=$i \
.venv/bin/vllm serve "$MODEL" --host 0.0.0.0 --port $((8000+i)) --tensor-parallel-size 1 \
--trust-remote-code --enable-prefix-caching --enforce-eager \
--trust-remote-code --enable-prefix-caching \
--dtype auto --gpu-memory-utilization 0.9 --max-model-len 200000 \
--kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_both"}' \
> "$OUTDIR/vllm_c_$i.log" 2>&1 &
@@ -40,7 +40,7 @@ done
echo "=== Launching PS instance on GPU 7, port 8007, bootstrap 9005 ==="
VLLM_MOONCAKE_BOOTSTRAP_PORT=9005 MASTER_PORT=29507 CUDA_VISIBLE_DEVICES=7 \
.venv/bin/vllm serve "$MODEL" --host 0.0.0.0 --port 8007 --tensor-parallel-size 1 \
--trust-remote-code --enable-prefix-caching --enforce-eager \
--trust-remote-code --enable-prefix-caching \
--dtype auto --gpu-memory-utilization 0.9 --max-model-len 200000 \
--kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_both"}' \
> "$OUTDIR/vllm_ps_0.log" 2>&1 &