gahow/aituner
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document 8-GPU harness ablation results for qwen27b and qwen235b prefill

Browse Source 
Add completed experiment results from dash0 runs after 2026-05-13:
- qwen27b chat 0-8k: harness +118.6% over no-harness (0.2696 vs 0.1233 req/s/GPU)
- qwen235b prefill TTFT 3s/6s/9s: harness +76.8% (0.3921 vs 0.2217 req/s/GPU)

Mark old 7-GPU and pre-5/13 docs as superseded. Update implementation
log with completed run status.
This commit is contained in:
Gahow Wang
2026-05-16 21:23:16 +08:00
parent d0c89dac48
commit 984eb1f325
5 changed files with 223 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/harness-ablation/profile-driven-harness-implementation-20260512.md
View File

@@ -141,8 +141,8 @@ New comparable studies:
| Variant | Study ID | Status |
| --- | --- | --- |
| no-harness baseline | `dash0-qwen27b-chat-0-8k-ttft4s-tpot25-gpu8-12iter-noharness-minprompt-gpt54-20260513` | running first |
| harness | `dash0-qwen27b-chat-0-8k-ttft4s-tpot25-gpu8-12iter-harness-profileplanner-20260513` | queued to run after baseline |
| no-harness baseline | `dash0-qwen27b-chat-0-8k-ttft4s-tpot25-gpu8-12iter-noharness-minprompt-gpt54-20260513` | completed |
| harness | `dash0-qwen27b-chat-0-8k-ttft4s-tpot25-gpu8-12iter-harness-profileplanner-20260513` | completed |
Both specs set:
@@ -152,4 +152,4 @@ Both specs set:
- SLO: TTFT p95 <= 4000ms, TPOT p95 <= 25ms, target pass rate 0.95
- search: full range, `inherit_incumbent_floor=false`
The no-harness baseline is running in tmux session `qwen27b-gpu8-noharness-20260513`. The harness run should only be started after the no-harness baseline finishes or reaches a sufficient early comparison point, because both need the full GPU host and should not run concurrently.
Results: harness best `0.2696 req/s/GPU` (TP=4, MBT=7680) vs no-harness best `0.1233 req/s/GPU` (prefix-caching=false), a **+118.6%** improvement. Full analysis in `qwen27b-chat-0-8k-ttft4s-tpot25-gpu8-20260513.md`.