Files
agentic-pd-hybrid/third_party/sglang/python/sglang/srt
kzlin ca4b64c79a feat(sglang): expose backpressure pause hint in admit_direct_append
Add `recommended_pause_ms` field to DirectAppendAdmissionReqOutput so D
can advise callers when its transfer queue is heavy or KV pool is near
capacity. The hint is computed from transfer_queue_depth,
retracted_queue_depth, and post-trim token_usage; thresholds are simple
heuristics (>0.90 usage, >=8 queue depth, retracted>0).

Default behavior is unchanged for callers that ignore the field.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 21:29:30 +08:00
..
2026-04-24 12:29:36 +00:00