bench: fix llama.cpp per-slot context (was 1/parallel of intended)

llama.cpp divides total -c across --parallel slots, so -c 4096 --parallel 4
gave each request only 1024 tokens — truncating long AIME generations before
the boxed answer and making xserv look artificially better (20% vs 3.3%).
Set total -c = max_seq_len * n_parallel so per-slot context equals xserv's
per-sequence max_seq_len. Also drop --log-disable; its startup log reports the
per-slot n_ctx that catches exactly this misconfiguration.

After the fix, AIME is at parity (xserv 23.3% vs llama.cpp 20.0%), matching the
GSM8K parity and confirming the gap was a config artifact, not engine quality.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-05-28 15:06:12 +08:00
parent 7cb9ee3870
commit 950ccf3822
2 changed files with 10 additions and 5 deletions

View File

@@ -134,7 +134,7 @@ def build_endpoints(args) -> list[SystemEndpoint]:
model_id=args.llama_model_id,
launch_cmd=llama_cpp_launch_cmd(
args.llama_bin, gguf, args.llama_port,
n_parallel=args.max_batch, ctx_size=args.max_seq_len,
n_parallel=args.max_batch, ctx_per_slot=args.max_seq_len,
),
# llama-server's health endpoint also returns 200 only when model is loaded.
health_path="/health",