Replace max/median hotspot index with (median, max) absolute pair
The max/median ratio inverts the actual user-facing p90 ranking:
sticky: hotspot=2.73 but system e2e p90 = 34.6s (worst)
unified: hotspot=3.67 but system e2e p90 = 18.0s (best)
because sticky's median is also high (everyone slow) while unified
concentrates the damage on one worker and keeps the other 7 fast.
Any "imbalance" metric structurally punishes the affinity-then-escape
schemes that we actually want to advocate for.
Changes:
- analysis/characterization/render_window1_figures.py:
fig_b3_per_worker_ttft now annotates each subplot with
"median X.Xs · max Y.Ys" instead of "hotspot=Y.YY"; docstring
documents why we drop the ratio.
- figs/f4c_per_worker_ttft.png: regenerated with new titles.
- figs/f4c_apc_vs_hotspot_tradeoff.png: deleted. The scatter's y-axis
was the deprecated ratio; superseded by f4c per-worker bars + f6
e2e bars which together carry the same information honestly.
- PAPER_OUTLINE.md: C3, §3.3, §4.1 wording, §5 metric list, §8
conclusion — replace "hotspot index" mentions with
"worst-worker p90" or "(median, max) worker p90"; promote the
§3.3 methodology note to a top-level sub-finding ("hot pin
failure must be measured with per-worker absolute latency,
not normalized ratio").
- MEETING.md: §3.3 narrative reworded to lead with the (median, max)
pair directly; explicit one-line note on why the ratio is dropped.
Conceptual uses of "hot session" / "hot instance" / "hot pin" remain
unchanged — only the *metric* called hotspot index is retired.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -147,7 +147,13 @@ def fig_b3_failure_breakdown(comp: dict, out: Path) -> None:
|
||||
|
||||
|
||||
def fig_b3_per_worker_ttft(results_dir: Path, comp: dict, out: Path) -> None:
|
||||
"""Per-worker TTFT p90 grouped bars; reads each policy's hotspot_index.json."""
|
||||
"""Per-worker TTFT p90 grouped bars; title shows median + max worker p90.
|
||||
|
||||
We deliberately do NOT report a max/median 'hotspot index' here: it is a
|
||||
ratio and treats unified (most workers fast, one hot) as worse than
|
||||
sticky (all workers slow), which inverts the actual user-facing p90.
|
||||
"""
|
||||
import statistics
|
||||
by = {r["policy"]: r for r in comp["rows"]}
|
||||
pols = [p for p in POLICY_ORDER if p in by]
|
||||
fig, axes = plt.subplots(1, len(pols), figsize=(3 * len(pols), 4),
|
||||
@@ -168,8 +174,12 @@ def fig_b3_per_worker_ttft(results_dir: Path, comp: dict, out: Path) -> None:
|
||||
edgecolor="black", linewidth=0.5)
|
||||
for i, v in enumerate(vals):
|
||||
ax.text(i, v, f"{v:.1f}", ha="center", va="bottom", fontsize=8)
|
||||
ax.set_title(f"{pol}\nhotspot={by[pol]['hotspot_index_ttft_p90']:.2f}",
|
||||
fontsize=10)
|
||||
median_v = statistics.median(vals)
|
||||
max_v = max(vals)
|
||||
ax.set_title(
|
||||
f"{pol}\nmedian {median_v:.1f}s · max {max_v:.1f}s",
|
||||
fontsize=10,
|
||||
)
|
||||
ax.tick_params(axis="x", labelsize=8)
|
||||
ax.grid(alpha=0.3, axis="y")
|
||||
axes[0].set_ylabel("worker TTFT p90 (s)")
|
||||
|
||||
Reference in New Issue
Block a user