gahow/agentic-kvc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Replayer: closed-loop inter-turn think-time mode

Browse Source 
Add --inter-turn-think (env REPLAY_INTER_TURN_THINK_S): turn 1 fires on
session admission, each later turn a FIXED think-time after the previous
turn COMPLETES, ignoring absolute trace timestamps. Combined with
--max-inflight-sessions (env REPLAY_MAX_INFLIGHT) this is a stable N-user
closed loop, removing the open-loop "fire immediately because timestamp is
in the past" retrigger artifact. Needed for the dispatch-coupling
(wall-clock amplification) sweep.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Gahow Wang
2026-05-29 18:19:12 +08:00
parent 657cd36f3d
commit 48ae72467a
2 changed files with 30 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
replayer/replay.py
View File

@@ -60,6 +60,12 @@ class ReplayConfig:
request_limit: int | None = None
model_name: str = "default"
max_inflight_sessions: int | None = None # cap on concurrent sessions; None = unlimited
# Closed-loop think-time mode: if set, ignore absolute trace timestamps for
# subsequent turns — fire turn 1 on session admission, then each later turn a
# FIXED think-time after the previous turn COMPLETES. Combined with
# max_inflight_sessions=N this is a stable N-user closed-loop (no open-loop
# runaway), so it removes the "immediate retrigger under load" artifact.
inter_turn_think_s: float | None = None
def _build_prompt_token_ids(req: TraceRequest) -> list[int]:
@@ -279,12 +285,19 @@ async def _run_session(
await session_sem.acquire()
realized_context: list[int] = []
try:
for req in state.turns:
# Wait until this request's trace timestamp
target_wall = (req.timestamp_s - earliest_ts)
elapsed = time.perf_counter() - sweep_start
if elapsed < target_wall:
await asyncio.sleep(target_wall - elapsed)
for turn_idx, req in enumerate(state.turns):
if config.inter_turn_think_s is not None:
# Closed-loop: turn 1 fires on admission; later turns wait a fixed
# think-time AFTER the previous turn completed (no absolute schedule,
# so no "fire immediately because timestamp is in the past").
if turn_idx > 0:
await asyncio.sleep(config.inter_turn_think_s)
else:
# Original: dispatch at the request's absolute trace timestamp.
target_wall = (req.timestamp_s - earliest_ts)
elapsed = time.perf_counter() - sweep_start
if elapsed < target_wall:
await asyncio.sleep(target_wall - elapsed)
token_ids = _apply_realized_prefix(
_build_prompt_token_ids(req),