## TODO - [x] 什么是 peak/valley，补一个 1 一天的 workload 波动 - [ ] 需要跨小时（+1h）的 trace 相似性 - [ ] 「需要 tune 多久？prefix trace window 的有效性」：这一部分需要跑一个 30min 的实验 - [ ] evaluator-reliability-compare 加一个 valley setup 的比较 [5/10] ```bash # probe qwen235b 的 trace sample threshold # [x] cd /home/admin/cpfs/wjh/aituner/tuner-workload-principle && ./start_qwen235b_tp4dp1_threshold_dash0123_tmux.sh # [x] bash /home/admin/cpfs/wjh/aituner/tuner-workload-principle/start_qwen235b_threshold_refresh_step1_dash0123_tmux.sh # 等 step1 全部完成后 # [x] bash /home/admin/cpfs/wjh/aituner/tuner-workload-principle/start_qwen235b_threshold_experiment_step2_dash0123_tmux.sh # 上面 qwen235b 10 parallel configs 的实验已经跑完，汇总观察到了 paper/workload_pattern_to_config_principles.md # [x] bash /home/admin/cpfs/wjh/aituner/tuner-workload-principle/start_qwen30b_0311_peak_valley_19parallel_threshold_chat_dash0123_tmux.sh # 跑完 merge bash /home/admin/cpfs/wjh/aituner/tuner-workload-principle/tmp/qwen30b_0311_peak_valley_19parallel_threshold_chat_dash0123_ts100/merge_results_v2_trace_tables.sh # TODO: 让 codex 跑 threshold 版本的 synthetic/semi-real # done # Ongoing # [x] bash /home/admin/cpfs/wjh/aituner/tuner-workload-principle/start_qwen235b_0311_peak_threshold_batching_tp4dp1_epoff19_dash0123_tmux.sh # decode-only # TBD MODEL=qwen30b bash ./start_decode_peak_parallel_sweep_dash0123_tmux.sh # going MODEL=qwen235b bash ./start_decode_peak_parallel_sweep_dash0123_tmux.sh # Ongoing # [x] bash ./start_decode_peak_thresholds_dash0123_tmux.sh # MODEL=qwen30b bash ./search_decode_peak_thresholds.sh # MODEL=qwen235b bash ./search_decode_peak_thresholds.sh #### TBD # qwen3-coder 的 19 parallel configs smoke test cd /home/admin/cpfs/wjh/aituner/tuner-workload-principle tmux new -s codernext_parallel_smoke 'bash ./start_qwen_coder_next_internal_parallel_only_smoke.sh' ``` qwen30b: chat: 19 parallel configs x 5 days coder: 19 parallel configs x 5 days qwen235b: chat: 10 parallel configs x 5 days coder: 10 parallel configs x 5 days ✅：TP4DP1EPOFF, TP8DP1EPOFF TP4DP2EPOFF - L 现在是 3 维 - log_mean_raw_len - log_p95_over_mean_raw_len - cv_raw_len - C 现在是 4 维 - log_mean_hit_len - log_p95_over_mean_hit_len - cv_hit_len - cache_saving_ratio - A 现在是 3 维 - log_qps - cv_interarrival - log_fano_1s_request_counts ## 一句话 principles prefill 时总是应该开 DP=1 prefill 时几乎 EP off > EP on，需要找到 EP on > EP off 的 case 分析 ## prefill node - 证明 config 对性能影响大/不同 workload 性能最优点不同：`data/qwen30b-config-performance-spread-v1.csv` qwen30b 361 configs (19 parallel x 19 batching (LPT=20480/32768)) chat/coder 0311 10:00~10:03 timescale/GPU=0.5 linear SLO 0.001L + 1.0 - workload 的跨天相似性：`data/weekday-workload-similarity.csv` 0311~0317 5 weekdays chat/coder, peak/valley (10:00~10:30/22:00~22:30) - tuned top5 configs 与未来的相似度，`data/qwen30b-high-perf-configs-jaccard.csv` qwen30b 19 configs (parallel) 0311~0317 5 weekdays, chat/coder, peak/valley (10:00~10:10/22:00~22:10) timescale/GPU=0.5 SLO: 0;8k;32k;=2s;4s;6s - tuned best configs 在未来能达到相对 oracle 的性能，`data/qwen30b-tuned-best-config-perf-across-5days.csv` qwen30b 19 configs (parallel) 0311~0317 5 weekdays, chat/coder, peak/valley (10:00~10:10/22:00~22:10) timescale/GPU=0.5 SLO: 0;8k;32k;=2s;4s;6s - synthetic/semi-real/real 对比，`data/qwen30b-evaluator-reliability-compare.csv` qwen30b 19 configs (parallel) 0311~0317 5 weekdays, chat/coder, peak (10:00~10:10) timescale/GPU=0.5 SLO: 0;8k;32k;=2s;4s;6s - 需要 tune 多久？prefix trace window 的有效性 `data/qwen30b-prefix-trace-window-stability.csv` qwen30b 19 configs (parallel) 0311~0317 5 weekdays, chat/coder, peak/valley (10:00~10:10/22:00~22:10) timescale/GPU=0.5 SLO: 0;8k;32k;=1s;2s;4s ## decode node TBD ## Similarity 计算 这张图的算法，直接看这两个 Python 文件就行： - 主脚本: [plot_similarity_heatmap_custom_windows.py](/home/admin/cpfs/wjh/aituner/tuner-workload-principle/workload-compare/plot_similarity_heatmap_custom_windows.py) - 归一化定义: [compute_signatures.py](/home/admin/cpfs/wjh/aituner/tuner-workload-principle/workload-compare/compute_signatures.py) - trace 排序与 catalog: [trace_catalog.py](/home/admin/cpfs/wjh/aituner/tuner-workload-principle/workload-compare/trace_catalog.py) 如果你问的是图里这张 `Similarity: 10:00-10:30 / 22:00-22:30`，核心算法就在： - 指标提取: [plot_similarity_heatmap_custom_windows.py:132](/home/admin/cpfs/wjh/aituner/tuner-workload-principle/workload-compare/plot_similarity_heatmap_custom_windows.py#L132) - 全局 robust normalization: [compute_signatures.py:64](/home/admin/cpfs/wjh/aituner/tuner-workload-principle/workload-compare/compute_signatures.py#L64) - 相似度矩阵: [plot_similarity_heatmap_custom_windows.py:247](/home/admin/cpfs/wjh/aituner/tuner-workload-principle/workload-compare/plot_similarity_heatmap_custom_windows.py#L247) **它实际怎么算** 1. 枚举所有 trace 来自 [trace_catalog.py:77](/home/admin/cpfs/wjh/aituner/tuner-workload-principle/workload-compare/trace_catalog.py#L77)，会收集： - `chat peak` - `chat valley` - `coder peak` - `coder valley` 并按 `trace_family, date, day_part` 排序。 2. 对每个 trace，截取指定窗口 在 [plot_similarity_heatmap_custom_windows.py:179](/home/admin/cpfs/wjh/aituner/tuner-workload-principle/workload-compare/plot_similarity_heatmap_custom_windows.py#L179)。 对 peak trace 用你传的 `10:00-10:30`，对 valley trace 用 `22:00-22:30`。 3. 对每个窗口算 5 维原始特征 在 [plot_similarity_heatmap_custom_windows.py:168](/home/admin/cpfs/wjh/aituner/tuner-workload-principle/workload-compare/plot_similarity_heatmap_custom_windows.py#L168)： - `load_tokens_per_s = total_input_tokens / total_duration_seconds` - `mean_input_length = mean(input_lengths)` - `p95_input_length = quantile(input_lengths, 0.95)` - `input_length_cv = std(input_lengths) / mean_input_lengths` - `burstiness = std(inter_arrivals) / mean(inter_arrivals)` 4. 把所有窗口放在一起，按每一维做全局 robust normalization 在 [compute_signatures.py:64](/home/admin/cpfs/wjh/aituner/tuner-workload-principle/workload-compare/compute_signatures.py#L64)： - 对每个维度 `x` - 计算所有窗口上的 `median, q1, q3` - `iqr = q3 - q1` - `global_z_x = (x - median) / iqr` - 如果 `iqr <= 0`，就强制设为 `1.0` 5. 每个窗口得到一个 5 维 normalized 向量 在 [plot_similarity_heatmap_custom_windows.py:248](/home/admin/cpfs/wjh/aituner/tuner-workload-principle/workload-compare/plot_similarity_heatmap_custom_windows.py#L248)： ```python [ global_z_load_tokens_per_s, global_z_mean_input_length, global_z_p95_input_length, global_z_input_length_cv, global_z_burstiness, ] ``` 6. 两两计算欧氏距离，再映射成相似度 在 [plot_similarity_heatmap_custom_windows.py:247](/home/admin/cpfs/wjh/aituner/tuner-workload-principle/workload-compare/plot_similarity_heatmap_custom_windows.py#L247)： - `distance(i, j) = ||v_i - v_j||_2` - `similarity(i, j) = exp(-distance(i, j))` 所以这张图的数学定义就是： ```text G(w) = [ z(load_tokens_per_s), z(mean_input_length), z(p95_input_length), z(input_length_cv), z(burstiness), ] z(x) = (x - median(x_all_windows)) / IQR(x_all_windows) d(a,b) = ||G(a) - G(b)||_2 sim(a,b) = exp(-d(a,b)) ``` 这里 `x_all_windows` 指的是这次参与画图的全部窗口，不是单个 window 内部。 **伪代码** ```python specs = build_trace_catalog() # chat/coder x peak/valley x all days rows = [] for spec in specs: if spec.day_part == "peak": window = peak_window # 10:00-10:30 else: window = valley_window # 22:00-22:30 reqs = load_requests_in_window(spec.trace_path, window) input_lengths = [r.input_length for r in reqs] inter_arrivals = diff([r.timestamp for r in reqs]) row = { "trace_family": spec.trace_family, "day_part": spec.day_part, "date": spec.date, "load_tokens_per_s": sum(input_lengths) / window_duration_sec, "mean_input_length": mean(input_lengths), "p95_input_length": p95(input_lengths), "input_length_cv": std(input_lengths) / mean(input_lengths), "burstiness": std(inter_arrivals) / mean(inter_arrivals), } rows.append(row) frame = DataFrame(rows) for col in [ "load_tokens_per_s", "mean_input_length", "p95_input_length", "input_length_cv", "burstiness", ]: med = median(frame[col]) iqr = p75(frame[col]) - p25(frame[col]) if iqr <= 0: iqr = 1.0 frame[f"global_z_{col}"] = (frame[col] - med) / iqr vectors = frame[[global_z_5_dims]].to_numpy() for i in range(len(vectors)): for j in range(len(vectors)): dist[i,j] = l2_norm(vectors[i] - vectors[j]) sim[i,j] = exp(-dist[i,j]) plot_heatmap(sim) ``` **补充两点** - 这张图不是用 `SIGNATURE_WEIGHTS` 画的。`0.35/0.2/...` 那组权重只用于 `signature_score`，不用于 heatmap 两两相似度。 - 这张图用的是 `global_robust_scale`，不是按 `chat/coder` 分开缩放，也不是每个窗口自己内部缩放。 如果你要，我下一步可以直接把这张图对应的命令行也给你写出来，并把它产出的 `metrics csv / normalization csv / similarity matrix csv` 一一对应到文件。 ## Semi-real 定义 - 到达过程：按 source trace 的平均 req_rate 生成 Poisson arrivals 代码在 prepare_figure08_evaluator_assets.py (line 115) 和 prepare_figure08_evaluator_assets.py (line 537) - 请求数：不固定，取决于这次 Poisson 过程实际生成了多少个 arrival - 长度分布：对每个生成的 arrival，从 source real trace 的请求列表里“有放回随机采样”一个 source request 代码在 prepare_figure08_evaluator_assets.py (line 272) - 被继承的字段：input_length、output_length、turn/type - 不保留原请求身份：hash_ids 会重新生成，不是原 real trace 的 hash_ids 代码在 prepare_figure08_evaluator_assets.py (line 169)