Files
agentic-pd-hybrid/third_party/sglang/python/sglang
Claude Code Agent a369722efe fix(sglang): account snapshot-reserved slots in radix mem leak check
Phase 2 prepare_receive allocates kv_pool slots that aren't visible
to radix / session bookkeeping until finalize_ingest. Without this
fix, the scheduler's idle self_check fires:

  ValueError: token_to_kv_pool_allocator memory leak detected!
    available=288391, evictable=5, protected=0, session_held=0
    (expected sum == 288460)

_check_radix_cache_memory now subtracts
  sum(len(rec.slot_indices) for rec in ctrl._ingest_records.values())
from the expected total before flagging a leak. Snapshot_reserved is
also printed in the leak message for diagnostics.

Smoke confirmed (scripts/smoke_snapshot_sglang_integration.py):
  [smoke] prepare_receive on P → 200: ok=true (96 layer bufs)
  [smoke] dump on D → 200: ok=false, reason=session-not-resident
  [smoke] finalize on P → 200: ok=true, inserted_prefix_len=0
  [smoke] OVERALL: PASS

End-to-end KV-correctness (snapshot ingest yields cache hit on next
prefill) still requires the agentic+router stack — covered in the E4
sweep, not this smoke.
2026-05-13 08:26:16 +08:00
..
2026-04-24 12:29:36 +00:00
2026-04-24 12:29:36 +00:00

Code Structure

  • eval: The evaluation utilities.
  • lang: The frontend language.
  • multimodal_gen: Inference framework for accelerated image/video generation.
  • srt: The backend engine for running local models. (SRT = SGLang Runtime).
  • test: The test utilities.
  • api.py: The public APIs.
  • bench_offline_throughput.py: Benchmark the performance in the offline mode.
  • bench_one_batch.py: Benchmark the latency of running a single static batch without a server.
  • bench_one_batch_server.py: Benchmark the latency of running a single batch with a server.
  • bench_serving.py: Benchmark online serving with dynamic requests.
  • check_env.py: Check the environment variables and dependencies.
  • global_config.py: The global configs and constants.
  • launch_server.py: The entry point for launching a local server.
  • profiler.py: The profiling entry point to send profile requests.
  • utils.py: Common utilities.
  • version.py: Version info.