gahow/agentic-kvc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
d9cf3126c6ce454b9435023ad9ecdc4933c1d117
agentic-kvc/analysis/characterization/elastic_migration_v2/data/breakdown_unified.json
Gahow Wang d76eb02637 Elastic migration v2 section: PD-sep on agentic workload is net negative 
New analysis/characterization/elastic_migration_v2/ packages the
unified_v2 + unified_kv_both experiments into a self-contained
results section that the paper can cite as the "we tried selective
PD-sep migration" case study. The section finds three independent
reasons PD-sep doesn't help on agentic w600:

1. Mooncake kv_both substrate alone (no PD-sep ever firing) imposes
   TTFT p90 +45%, TPOT p90 +25%, hotspot index +19% vs plain
   unified. Per-step KVConnectorMetadata maintenance and block
   reservation semantics dominate even when no transfer is pending.
2. PD-sep gate fires only 0.16-0.41% of requests across two
   gate-tightness configurations. 88-76% are killed by
   new_local < threshold because 93% intra-session reuse on agentic
   traces leaves a small uncached tail; 19% are killed by
   chosen_no_active_decode (snapshot-time gate). Even relaxed
   thresholds can't grow trigger rate past 0.5%.
3. When PD-sep fires, the calibrated cost model
   (0.3s + bytes / 2.7 GB/s) is wrong by 10-20x. 5 triggered
   requests in v2.1 saw realized TTFT 12-45s vs model-predicted
   migrate cost 0.7-2.2s, consistent with the E2 audit's finding
   that D-side block pre-reservation and missing layerwise
   pipelining dominate the decode_sent -> first_token clock.

Three-way comparison (unified vs unified_kv_both vs unified_v2):
v2 vs the kv_both control is roughly net-zero (-10% hotspot,
-14% TPOT p90, +3% TTFT p90, +9% TTFT p99). v2 vs plain unified is
strictly worse by 27-49% across latency percentiles because the
kv_both substrate tax is unavoidable when the policy is enabled.

Contents:
- README.md: the four results sections, the three-way comparison
  table, an explicit "what this claims for the paper" list, and a
  cross-reference index to the earlier characterization documents.
- data/: b3_policy_comparison.json + per-policy breakdown.json
  + per-policy hotspot_index.json for the four policies in scope.
- figures/: 4 PNGs rendered by render_figures.py:
  * fig_kv_both_overhead.png   — 4-metric bar chart with delta
    annotations showing kv_both alone costs +45% TTFT p90.
  * fig_v2_trigger_funnel.png  — per-reason request count for the
    two gate configurations on log scale.
  * fig_v2_predicted_vs_actual.png  — scatter of model-predicted
    migrate cost vs realized TTFT for the 5 triggered requests,
    with y=x, 10x, and 20x reference lines.
  * fig_three_way_hotspot.png  — per-worker TTFT p90 grouped bars
    across the three policies.

The section is intentionally self-contained: it lists what the
experiment validates (cost model picks correct candidates;
shadow-drift fix is necessary; same-worker interference is real)
alongside what it disproves (per-request PD-sep on agentic via
Mooncake is not a net win in current implementation).

Refs: E1/E2 subagent audits, B2 microbench, unified_v2 commits
19f69a9 / 4b833d3 / 95c8ef8.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-26 13:28:37 +08:00

1 line
3.1 MiB
Raw Blame History

[{"request_id":"1237198:1:1237198:0","session_id":"1237198","input_length":8228,"t_proxy_recv":281548.676951257,"t_decision_unix":1779712413.690889,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8228,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281549.633507788,"t_first_token_unix":1779712414.6474519,"t_done":281549.728127214,"t_done_unix":1779712414.7420704},{"request_id":"1237337:1:1237337:1","session_id":"1237337","input_length":3976,"t_proxy_recv":281549.095120592,"t_decision_unix":1779712414.1090617,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3976,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":8228,"ongoing_decode_tokens":0,"pending_prefill_tokens":8228,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":8228.0,"score_lmetric":12204},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281549.666723763,"t_first_token_unix":1779712414.6806674,"t_done":281550.059262737,"t_done_unix":1779712415.0732064},{"request_id":"1237198:2:1237773:2","session_id":"1237198","input_length":10237,"t_proxy_recv":281550.644691503,"t_decision_unix":1779712415.6586328,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":8192,"affinity_cache_ratio":0.8002344436846732,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":8192,"estimated_new_tokens":2045,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":16,"cache_hit":8192,"new_prefill":2045,"score_linear":-8192.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":7,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-8192.0,"chosen_score_lmetric":0,"t_first_token":281550.948811384,"t_first_token_unix":1779712415.962755,"t_done":281551.082256897,"t_done_unix":1779712416.0962},{"request_id":"1237877:1:1237877:3","session_id":"1237877","input_length":749,"t_proxy_recv":281551.101977294,"t_decision_unix":1779712416.1159189,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":749,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":7,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281551.24527805,"t_first_token_unix":1779712416.2592218,"t_done":281551.37381505,"t_done_unix":1779712416.3877585},{"request_id":"1239034:1:1239034:4","session_id":"1239034","input_length":13489,"t_proxy_recv":281554.880719724,"t_decision_unix":1779712419.894661,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":13489,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":8,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281556.417671804,"t_first_token_unix":1779712421.4316158,"t_done":281556.868602104,"t_done_unix":1779712421.8825452},{"request_id":"1237877:2:1239577:6","session_id":"1237877","input_length":1620,"t_proxy_recv":281556.764405302,"t_decision_unix":1779712421.7783465,"policy":"unified","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":512,"affinity_cache_ratio":0.3160493827160494,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":512,"estimated_new_tokens":1108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":1620,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":8,"cache_hit":512,"new_prefill":1108,"score_linear":-512.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":13489,"ongoing_decode_tokens":13489,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":13489.0,"score_lmetric":1620},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":0,"pending_prefill_tokens":37165,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":37165.0,"score_lmetric":38785},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-512.0,"chosen_score_lmetric":0,"t_first_token":281556.84530383,"t_first_token_unix":1779712421.8592472,"t_done":281557.296524878,"t_done_unix":1779712422.3104684},{"request_id":"1239704:1:1239704:7","session_id":"1239704","input_length":5194,"t_proxy_recv":281557.206812351,"t_decision_unix":1779712422.2207534,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5194,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":1620,"ongoing_decode_tokens":1620,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":8,"cache_hit":0,"new_prefill":5194,"score_linear":1620.0,"score_lmetric":5194},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":0,"pending_prefill_tokens":37165,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":5194,"score_linear":37165.0,"score_lmetric":42359},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281557.85556446,"t_first_token_unix":1779712422.869508,"t_done":281558.12856737,"t_done_unix":1779712423.142511},{"request_id":"1240924:1:1240924:11","session_id":"1240924","input_length":7429,"t_proxy_recv":281561.228395146,"t_decision_unix":1779712426.2423363,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7429,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":7429,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":7429,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7429,"score_linear":66214.0,"score_lmetric":73643},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":30040,"ongoing_decode_tokens":0,"pending_prefill_tokens":30040,"num_requests":1,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":7429,"score_linear":30040.0,"score_lmetric":37469},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":7429,"score_linear":18420.0,"score_lmetric":7429},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7429,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":37165,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7429,"score_linear":37165.0,"score_lmetric":7429},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7429,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281561.769135984,"t_first_token_unix":1779712426.7830799,"t_done":281561.949931507,"t_done_unix":1779712426.9638746},{"request_id":"1239100:1:1239100:5","session_id":"1239100","input_length":37165,"t_proxy_recv":281555.121400319,"t_decision_unix":1779712420.1353414,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":37165,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":8,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":13489,"ongoing_decode_tokens":0,"pending_prefill_tokens":13489,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":13489.0,"score_lmetric":50654},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281561.145798439,"t_first_token_unix":1779712426.1597428,"t_done":281562.412950523,"t_done_unix":1779712427.4268942},{"request_id":"1240198:1:1240198:9","session_id":"1240198","input_length":30040,"t_proxy_recv":281558.742521259,"t_decision_unix":1779712423.7564623,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":30040,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":30040,"score_linear":66214.0,"score_lmetric":96254},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":0,"pending_prefill_tokens":37165,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":30040,"score_linear":37165.0,"score_lmetric":67205},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281562.676223903,"t_first_token_unix":1779712427.6901672,"t_done":281563.326382588,"t_done_unix":1779712428.3403256},{"request_id":"1241953:1:1241953:13","session_id":"1241953","input_length":1093,"t_proxy_recv":281564.644509509,"t_decision_unix":1779712429.6584508,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1093,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":1093,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":34,"cache_hit":0,"new_prefill":1093,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1093,"score_linear":66214.0,"score_lmetric":67307},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":68,"cache_hit":0,"new_prefill":1093,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":1093,"score_linear":18420.0,"score_lmetric":1093},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1093,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7587,"ongoing_decode_tokens":0,"pending_prefill_tokens":7587,"num_requests":1,"active_p_offloads":0,"cached_blocks":70,"cache_hit":0,"new_prefill":1093,"score_linear":7587.0,"score_lmetric":8680},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1093,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281564.760699374,"t_first_token_unix":1779712429.774643,"t_done":281565.281838493,"t_done_unix":1779712430.2957816},{"request_id":"1241942:1:1241942:12","session_id":"1241942","input_length":7587,"t_proxy_recv":281564.599081972,"t_decision_unix":1779712429.6130233,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7587,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":34,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7587,"score_linear":66214.0,"score_lmetric":73801},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":68,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":7587,"score_linear":18420.0,"score_lmetric":7587},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":70,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281565.128051933,"t_first_token_unix":1779712430.1419954,"t_done":281565.319627429,"t_done_unix":1779712430.3335705},{"request_id":"1237877:3:1242953:15","session_id":"1237877","input_length":2659,"t_proxy_recv":281568.032966231,"t_decision_unix":1779712433.0469077,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":1536,"affinity_cache_ratio":0.5776607747273411,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":1123,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":17157,"ongoing_decode_tokens":0,"pending_prefill_tokens":17157,"num_requests":1,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":2659,"score_linear":17157.0,"score_lmetric":19816},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":2659,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":2659,"score_linear":66214.0,"score_lmetric":68873},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":68,"cache_hit":1536,"new_prefill":1123,"score_linear":-1536.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":2659,"score_linear":18420.0,"score_lmetric":2659},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":2659,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":2659,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":2659,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":281568.12630083,"t_first_token_unix":1779712433.140244,"t_done":281568.190593472,"t_done_unix":1779712433.204537},{"request_id":"1242838:1:1242838:14","session_id":"1242838","input_length":17157,"t_proxy_recv":281567.651720821,"t_decision_unix":1779712432.665662,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":17157,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":17157,"score_linear":66214.0,"score_lmetric":83371},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":68,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":17157,"score_linear":18420.0,"score_lmetric":17157},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281569.347720435,"t_first_token_unix":1779712434.3616645,"t_done":281570.246312228,"t_done_unix":1779712435.2602556},{"request_id":"1239704:2:1243943:17","session_id":"1239704","input_length":14812,"t_proxy_recv":281571.310430045,"t_decision_unix":1779712436.3243713,"policy":"unified","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":5120,"affinity_cache_ratio":0.3456656764785309,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":5120,"estimated_new_tokens":9692,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":43,"cache_hit":5120,"new_prefill":9692,"score_linear":-5120.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":11074,"ongoing_decode_tokens":0,"pending_prefill_tokens":11074,"num_requests":1,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":14812,"score_linear":11074.0,"score_lmetric":25886},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":14812,"score_linear":66214.0,"score_lmetric":81026},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":70,"cache_hit":0,"new_prefill":14812,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":14812,"score_linear":18420.0,"score_lmetric":14812},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":14812,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":14812,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":14812,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-5120.0,"chosen_score_lmetric":0,"t_first_token":281572.398470817,"t_first_token_unix":1779712437.4124143,"t_done":281572.509673378,"t_done_unix":1779712437.5236166},{"request_id":"1243831:1:1243831:16","session_id":"1243831","input_length":11074,"t_proxy_recv":281570.908195976,"t_decision_unix":1779712435.9221373,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":11074,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":43,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":11074,"score_linear":66214.0,"score_lmetric":77288},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":70,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":11074,"score_linear":18420.0,"score_lmetric":11074},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281571.846067502,"t_first_token_unix":1779712436.860011,"t_done":281573.319884905,"t_done_unix":1779712438.3338284},{"request_id":"1239755:1:1239755:8","session_id":"1239755","input_length":66214,"t_proxy_recv":281557.422949382,"t_decision_unix":1779712422.4368904,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":66214,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5194,"ongoing_decode_tokens":0,"pending_prefill_tokens":5194,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":66214,"score_linear":5194.0,"score_lmetric":71408},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":0,"pending_prefill_tokens":37165,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":66214,"score_linear":37165.0,"score_lmetric":103379},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281573.306643605,"t_first_token_unix":1779712438.3205872,"t_done":281573.997266805,"t_done_unix":1779712439.01121},{"request_id":"1239034:2:1240558:10","session_id":"1239034","input_length":18420,"t_proxy_recv":281559.977755355,"t_decision_unix":1779712424.9916964,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":13312,"affinity_cache_ratio":0.7226927252985885,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":13312,"estimated_new_tokens":5108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":18420,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":18420,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":18420,"score_linear":66214.0,"score_lmetric":84634},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":30040,"ongoing_decode_tokens":0,"pending_prefill_tokens":30040,"num_requests":1,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":18420,"score_linear":30040.0,"score_lmetric":48460},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":13312,"new_prefill":5108,"score_linear":-13312.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":18420,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":0,"pending_prefill_tokens":37165,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":18420,"score_linear":37165.0,"score_lmetric":55585},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":18420,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-13312.0,"chosen_score_lmetric":0,"t_first_token":281560.697760618,"t_first_token_unix":1779712425.7117045,"t_done":281574.169393924,"t_done_unix":1779712439.1833372},{"request_id":"1244640:1:1244640:18","session_id":"1244640","input_length":8274,"t_proxy_recv":281573.803457265,"t_decision_unix":1779712438.8173988,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8274,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":66214,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8274,"score_linear":66214.0,"score_lmetric":8274},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":70,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":8274,"score_linear":18420.0,"score_lmetric":8274},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281574.396988638,"t_first_token_unix":1779712439.4109325,"t_done":281574.601444069,"t_done_unix":1779712439.6153874},{"request_id":"1241953:2:1245624:20","session_id":"1241953","input_length":10382,"t_proxy_recv":281577.028059827,"t_decision_unix":1779712442.0420012,"policy":"unified","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":1024,"affinity_cache_ratio":0.09863224812174919,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":1024,"estimated_new_tokens":9358,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":1024,"new_prefill":9358,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":6109,"ongoing_decode_tokens":0,"pending_prefill_tokens":6109,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10382,"score_linear":6109.0,"score_lmetric":16491}],"chosen_score_linear":-1024.0,"chosen_score_lmetric":0,"t_first_token":281577.799767373,"t_first_token_unix":1779712442.8137112,"t_done":281577.984341355,"t_done_unix":1779712442.9982843},{"request_id":"1245572:1:1245572:19","session_id":"1245572","input_length":6109,"t_proxy_recv":281576.937427708,"t_decision_unix":1779712441.951369,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":6109,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281577.974054792,"t_first_token_unix":1779712442.9879982,"t_done":281578.219838671,"t_done_unix":1779712443.2337818},{"request_id":"1245572:2:1246348:22","session_id":"1245572","input_length":6182,"t_proxy_recv":281579.533850205,"t_decision_unix":1779712444.5477917,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":6144,"affinity_cache_ratio":0.9938531219670009,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":6144,"estimated_new_tokens":38,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":17447,"ongoing_decode_tokens":0,"pending_prefill_tokens":10279,"num_requests":1,"active_p_offloads":0,"cached_blocks":76,"cache_hit":0,"new_prefill":6182,"score_linear":17447.0,"score_lmetric":16461},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":6144,"new_prefill":38,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-6144.0,"chosen_score_lmetric":0,"t_first_token":281579.564593299,"t_first_token_unix":1779712444.5785365,"t_done":281579.745293038,"t_done_unix":1779712444.759236},{"request_id":"1240924:2:1246244:21","session_id":"1240924","input_length":17447,"t_proxy_recv":281579.159806459,"t_decision_unix":1779712444.1737478,"policy":"unified","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":7168,"affinity_cache_ratio":0.4108442712214134,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":7168,"estimated_new_tokens":10279,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":76,"cache_hit":7168,"new_prefill":10279,"score_linear":-7168.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7168.0,"chosen_score_lmetric":0,"t_first_token":281580.371412753,"t_first_token_unix":1779712445.3853562,"t_done":281580.74416753,"t_done_unix":1779712445.7581112},{"request_id":"1241942:2:1247111:23","session_id":"1241942","input_length":17975,"t_proxy_recv":281582.207635924,"t_decision_unix":1779712447.221577,"policy":"unified","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":7168,"affinity_cache_ratio":0.3987760778859527,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":7168,"estimated_new_tokens":10807,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":7168,"new_prefill":10807,"score_linear":-7168.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7168.0,"chosen_score_lmetric":0,"t_first_token":281583.496957717,"t_first_token_unix":1779712448.5109026,"t_done":281583.68364178,"t_done_unix":1779712448.697586},{"request_id":"1244640:2:1247941:24","session_id":"1244640","input_length":10696,"t_proxy_recv":281585.022375379,"t_decision_unix":1779712450.0363166,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":8192,"affinity_cache_ratio":0.7658937920718025,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":8192,"estimated_new_tokens":2504,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":8192,"new_prefill":2504,"score_linear":-8192.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-8192.0,"chosen_score_lmetric":0,"t_first_token":281585.356168581,"t_first_token_unix":1779712450.3701117,"t_done":281585.401798313,"t_done_unix":1779712450.4157414},{"request_id":"1248768:1:1248768:25","session_id":"1248768","input_length":11515,"t_proxy_recv":281587.803580632,"t_decision_unix":1779712452.8175218,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":11515,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281588.785374426,"t_first_token_unix":1779712453.799318,"t_done":281589.048042416,"t_done_unix":1779712454.0619853},{"request_id":"1249058:1:1249058:26","session_id":"1249058","input_length":4126,"t_proxy_recv":281588.823933428,"t_decision_unix":1779712453.837875,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4126,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":11515,"ongoing_decode_tokens":11515,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":4126,"score_linear":11515.0,"score_lmetric":4126},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281589.090042453,"t_first_token_unix":1779712454.103986,"t_done":281589.257322876,"t_done_unix":1779712454.2712657},{"request_id":"1248768:2:1249765:27","session_id":"1248768","input_length":12188,"t_proxy_recv":281591.086875946,"t_decision_unix":1779712456.1008167,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":11264,"affinity_cache_ratio":0.924187725631769,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":924,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":83,"cache_hit":11264,"new_prefill":924,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":45,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11264.0,"chosen_score_lmetric":0,"t_first_token":281591.227392887,"t_first_token_unix":1779712456.2413363,"t_done":281591.696116404,"t_done_unix":1779712456.7100594},{"request_id":"1249058:2:1249872:28","session_id":"1249058","input_length":7482,"t_proxy_recv":281591.478563707,"t_decision_unix":1779712456.492505,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":4096,"affinity_cache_ratio":0.5474472066292435,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":4096,"estimated_new_tokens":3386,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":12188,"ongoing_decode_tokens":12188,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":83,"cache_hit":0,"new_prefill":7482,"score_linear":12188.0,"score_lmetric":7482},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":45,"cache_hit":4096,"new_prefill":3386,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-4096.0,"chosen_score_lmetric":0,"t_first_token":281591.798973446,"t_first_token_unix":1779712456.812917,"t_done":281591.927752112,"t_done_unix":1779712456.9416952},{"request_id":"1242838:2:1250503:29","session_id":"1242838","input_length":21578,"t_proxy_recv":281593.752334586,"t_decision_unix":1779712458.766276,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":16896,"affinity_cache_ratio":0.783019742330151,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":16896,"estimated_new_tokens":4682,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":16896,"new_prefill":4682,"score_linear":-16896.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-16896.0,"chosen_score_lmetric":0,"t_first_token":281594.456949715,"t_first_token_unix":1779712459.4708934,"t_done":281594.950881535,"t_done_unix":1779712459.9648247},{"request_id":"1250772:1:1250772:30","session_id":"1250772","input_length":6829,"t_proxy_recv":281594.613928148,"t_decision_unix":1779712459.6278694,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":6829,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":21578,"ongoing_decode_tokens":21578,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":6829,"score_linear":21578.0,"score_lmetric":6829},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281595.420193858,"t_first_token_unix":1779712460.4341369,"t_done":281597.018871056,"t_done_unix":1779712462.0328143},{"request_id":"1251059:1:1251059:32","session_id":"1251059","input_length":16512,"t_proxy_recv":281595.798423464,"t_decision_unix":1779712460.8123646,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":16512,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":93,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":6829,"ongoing_decode_tokens":6829,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":16512,"score_linear":6829.0,"score_lmetric":16512},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20086,"ongoing_decode_tokens":0,"pending_prefill_tokens":20086,"num_requests":1,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":16512,"score_linear":20086.0,"score_lmetric":36598},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281597.358333155,"t_first_token_unix":1779712462.3722763,"t_done":281597.360058486,"t_done_unix":1779712462.3740015},{"request_id":"1248768:3:1251261:33","session_id":"1248768","input_length":16989,"t_proxy_recv":281596.36989985,"t_decision_unix":1779712461.383841,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":11776,"affinity_cache_ratio":0.6931543940196597,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":5213,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":93,"cache_hit":11776,"new_prefill":5213,"score_linear":-11776.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":16989,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16512,"ongoing_decode_tokens":0,"pending_prefill_tokens":16512,"num_requests":1,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":16989,"score_linear":16512.0,"score_lmetric":33501},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":16989,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":16989,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":6829,"ongoing_decode_tokens":6829,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":16989,"score_linear":6829.0,"score_lmetric":16989},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20086,"ongoing_decode_tokens":0,"pending_prefill_tokens":20086,"num_requests":1,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":16989,"score_linear":20086.0,"score_lmetric":37075},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":16989,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":0,"t_first_token":281597.046204572,"t_first_token_unix":1779712462.060148,"t_done":281598.02276754,"t_done_unix":1779712463.0367105},{"request_id":"1251828:1:1251828:35","session_id":"1251828","input_length":1090,"t_proxy_recv":281598.299910945,"t_decision_unix":1779712463.3138523,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1090,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20086,"ongoing_decode_tokens":20086,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":1090,"score_linear":20086.0,"score_lmetric":1090},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":13780,"ongoing_decode_tokens":0,"pending_prefill_tokens":7636,"num_requests":1,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":1090,"score_linear":13780.0,"score_lmetric":8726}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281598.386755652,"t_first_token_unix":1779712463.400699,"t_done":281598.495429099,"t_done_unix":1779712463.5093722},{"request_id":"1250935:1:1250935:31","session_id":"1250935","input_length":20086,"t_proxy_recv":281595.239294734,"t_decision_unix":1779712460.2532363,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":20086,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":93,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":6829,"ongoing_decode_tokens":0,"pending_prefill_tokens":6829,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":20086,"score_linear":6829.0,"score_lmetric":26915},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281597.322810764,"t_first_token_unix":1779712462.3367543,"t_done":281598.578379352,"t_done_unix":1779712463.5923223},{"request_id":"1245572:3:1251662:34","session_id":"1245572","input_length":13780,"t_proxy_recv":281597.781024445,"t_decision_unix":1779712462.7949657,"policy":"unified","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":6144,"affinity_cache_ratio":0.4458635703918723,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":6144,"estimated_new_tokens":7636,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":16989,"ongoing_decode_tokens":16989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":93,"cache_hit":0,"new_prefill":13780,"score_linear":16989.0,"score_lmetric":13780},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":13780,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":13780,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":13780,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":13780,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":13780,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20086,"ongoing_decode_tokens":20086,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":13780,"score_linear":20086.0,"score_lmetric":13780},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":6144,"new_prefill":7636,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-6144.0,"chosen_score_lmetric":0,"t_first_token":281598.579468679,"t_first_token_unix":1779712463.593412,"t_done":281598.803988335,"t_done_unix":1779712463.8179314},{"request_id":"1252010:1:1252010:36","session_id":"1252010","input_length":9855,"t_proxy_recv":281598.908698508,"t_decision_unix":1779712463.9226396,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9855,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":144,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281599.737471785,"t_first_token_unix":1779712464.7514153,"t_done":281599.90678977,"t_done_unix":1779712464.9207335},{"request_id":"1248768:4:1253385:37","session_id":"1248768","input_length":17309,"t_proxy_recv":281603.682959978,"t_decision_unix":1779712468.6969006,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":16896,"affinity_cache_ratio":0.9761395805650239,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":16896,"estimated_new_tokens":413,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":103,"cache_hit":16896,"new_prefill":413,"score_linear":-16896.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-16896.0,"chosen_score_lmetric":0,"t_first_token":281603.763240432,"t_first_token_unix":1779712468.777184,"t_done":281606.733454465,"t_done_unix":1779712471.7473977},{"request_id":"1251828:2:1254272:41","session_id":"1251828","input_length":3934,"t_proxy_recv":281606.652419412,"t_decision_unix":1779712471.6663606,"policy":"unified","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":4,"affinity_cache_hit":1024,"affinity_cache_ratio":0.2602948652770717,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3934,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":17309,"ongoing_decode_tokens":17309,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":3934,"score_linear":17309.0,"score_lmetric":3934},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35221,"ongoing_decode_tokens":0,"pending_prefill_tokens":35221,"num_requests":1,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":3934,"score_linear":35221.0,"score_lmetric":39155},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":3934,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":1024,"new_prefill":2910,"score_linear":48600.0,"score_lmetric":52534},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":3934,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":10764,"ongoing_decode_tokens":0,"pending_prefill_tokens":4108,"num_requests":1,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":3934,"score_linear":10764.0,"score_lmetric":8042},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":3934,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":3934,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281606.896409357,"t_first_token_unix":1779712471.9103525,"t_done":281607.042768551,"t_done_unix":1779712472.0567117},{"request_id":"1254584:1:1254584:42","session_id":"1254584","input_length":748,"t_proxy_recv":281607.784331837,"t_decision_unix":1779712472.7982733,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":748,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":748,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35221,"ongoing_decode_tokens":0,"pending_prefill_tokens":35221,"num_requests":1,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":748,"score_linear":35221.0,"score_lmetric":35969},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":748,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":748,"score_linear":49624.0,"score_lmetric":50372},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":748,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":10764,"ongoing_decode_tokens":10764,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":748,"score_linear":10764.0,"score_lmetric":748},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":748,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":748,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281607.85029791,"t_first_token_unix":1779712472.8642414,"t_done":281608.004182564,"t_done_unix":1779712473.0181255},{"request_id":"1252010:2:1254691:43","session_id":"1252010","input_length":13092,"t_proxy_recv":281608.237398497,"t_decision_unix":1779712473.2513402,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":9728,"affinity_cache_ratio":0.743049190345249,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9728,"estimated_new_tokens":3364,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":13092,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35221,"ongoing_decode_tokens":0,"pending_prefill_tokens":35221,"num_requests":1,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":13092,"score_linear":35221.0,"score_lmetric":48313},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":13092,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":13092,"score_linear":49624.0,"score_lmetric":62716},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":13092,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":10764,"ongoing_decode_tokens":10764,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":13092,"score_linear":10764.0,"score_lmetric":13092},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":9728,"new_prefill":3364,"score_linear":-9728.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":13092,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9728.0,"chosen_score_lmetric":0,"t_first_token":281608.667539067,"t_first_token_unix":1779712473.6814826,"t_done":281608.835287652,"t_done_unix":1779712473.8492308},{"request_id":"1250772:2:1254198:40","session_id":"1250772","input_length":10764,"t_proxy_recv":281606.380243854,"t_decision_unix":1779712471.394185,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":6656,"affinity_cache_ratio":0.6183574879227053,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":6656,"estimated_new_tokens":4108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":17309,"ongoing_decode_tokens":17309,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":10764,"score_linear":17309.0,"score_lmetric":10764},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35221,"ongoing_decode_tokens":0,"pending_prefill_tokens":35221,"num_requests":1,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":10764,"score_linear":35221.0,"score_lmetric":45985},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":10764,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":10764,"score_linear":49624.0,"score_lmetric":60388},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":10764,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":6656,"new_prefill":4108,"score_linear":-6656.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":10764,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":10764,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-6656.0,"chosen_score_lmetric":0,"t_first_token":281606.763529758,"t_first_token_unix":1779712471.7774732,"t_done":281609.038341889,"t_done_unix":1779712474.0522852},{"request_id":"1253804:1:1253804:39","session_id":"1253804","input_length":35221,"t_proxy_recv":281605.271740625,"t_decision_unix":1779712470.2856817,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":35221,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":17309,"ongoing_decode_tokens":17309,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":35221,"score_linear":17309.0,"score_lmetric":35221},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":35221,"score_linear":49624.0,"score_lmetric":84845},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281610.440644839,"t_first_token_unix":1779712475.4545884,"t_done":281612.655323923,"t_done_unix":1779712477.6692672},{"request_id":"1256273:1:1256273:46","session_id":"1256273","input_length":1694,"t_proxy_recv":281613.467438811,"t_decision_unix":1779712478.4813805,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1694,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":1694,"score_linear":49624.0,"score_lmetric":51318},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":17786,"ongoing_decode_tokens":0,"pending_prefill_tokens":10618,"num_requests":1,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":1694,"score_linear":17786.0,"score_lmetric":12312},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":21,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281613.613233451,"t_first_token_unix":1779712478.6271765,"t_done":281613.898880937,"t_done_unix":1779712478.9128242},{"request_id":"1243831:2:1256324:47","session_id":"1243831","input_length":11448,"t_proxy_recv":281613.631723869,"t_decision_unix":1779712478.6456656,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":11264,"affinity_cache_ratio":0.983927323549965,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":184,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":11448,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":11264,"new_prefill":184,"score_linear":-11264.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":11448,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":11448,"score_linear":49624.0,"score_lmetric":61072},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":17786,"ongoing_decode_tokens":0,"pending_prefill_tokens":10618,"num_requests":1,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":11448,"score_linear":17786.0,"score_lmetric":22066},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":1694,"ongoing_decode_tokens":1694,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":21,"cache_hit":0,"new_prefill":11448,"score_linear":1694.0,"score_lmetric":11448},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":11448,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":11448,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11264.0,"chosen_score_lmetric":0,"t_first_token":281613.688090054,"t_first_token_unix":1779712478.7020328,"t_done":281614.12771778,"t_done_unix":1779712479.141661},{"request_id":"1253743:1:1253743:38","session_id":"1253743","input_length":49624,"t_proxy_recv":281605.029949971,"t_decision_unix":1779712470.0438905,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":49624,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":17309,"ongoing_decode_tokens":17309,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":49624,"score_linear":17309.0,"score_lmetric":49624},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281614.280846153,"t_first_token_unix":1779712479.2947896,"t_done":281614.681211634,"t_done_unix":1779712479.6951547},{"request_id":"1249058:3:1256222:45","session_id":"1249058","input_length":17786,"t_proxy_recv":281613.320932488,"t_decision_unix":1779712478.3348737,"policy":"unified","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":7168,"affinity_cache_ratio":0.4030136062071292,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":7168,"estimated_new_tokens":10618,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":17786,"score_linear":49624.0,"score_lmetric":67410},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":7168,"new_prefill":10618,"score_linear":-7168.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":21,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7168.0,"chosen_score_lmetric":0,"t_first_token":281614.583621443,"t_first_token_unix":1779712479.597565,"t_done":281614.70156554,"t_done_unix":1779712479.7155087},{"request_id":"1253743:2:1256070:44","session_id":"1253743","input_length":49687,"t_proxy_recv":281614.690198436,"t_decision_unix":1779712479.7041397,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9995371022601486,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":23,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":49687,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":3584,"new_prefill":46103,"score_linear":-3584.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":49687,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":49664,"new_prefill":23,"score_linear":-49664.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":17786,"ongoing_decode_tokens":17786,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":49687,"score_linear":17786.0,"score_lmetric":49687},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":24,"cache_hit":0,"new_prefill":49687,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":49687,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":49687,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":281614.763213676,"t_first_token_unix":1779712479.7771568,"t_done":281615.594039343,"t_done_unix":1779712480.6079826},{"request_id":"1256273:2:1257769:51","session_id":"1256273","input_length":6170,"t_proxy_recv":281618.451755825,"t_decision_unix":1779712483.465697,"policy":"unified","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":1536,"affinity_cache_ratio":0.24894651539708265,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":4634,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":4434,"ongoing_decode_tokens":4434,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":6170,"score_linear":4434.0,"score_lmetric":6170},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":27387,"ongoing_decode_tokens":0,"pending_prefill_tokens":27387,"num_requests":1,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":6170,"score_linear":27387.0,"score_lmetric":33557},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":6170,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":6170,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":6170,"score_linear":90488.0,"score_lmetric":96658},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":24,"cache_hit":1536,"new_prefill":4634,"score_linear":-1536.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":6170,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":6170,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":281618.775599913,"t_first_token_unix":1779712483.789544,"t_done":281619.079216777,"t_done_unix":1779712484.0931602},{"request_id":"1257572:1:1257572:50","session_id":"1257572","input_length":4434,"t_proxy_recv":281617.816522383,"t_decision_unix":1779712482.830464,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4434,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":27387,"ongoing_decode_tokens":0,"pending_prefill_tokens":27387,"num_requests":1,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":4434,"score_linear":27387.0,"score_lmetric":31821},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":4434,"score_linear":90488.0,"score_lmetric":94922},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":24,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281618.098634794,"t_first_token_unix":1779712483.1125782,"t_done":281619.340595381,"t_done_unix":1779712484.3545387},{"request_id":"1254584:2:1258039:52","session_id":"1254584","input_length":1257,"t_proxy_recv":281619.470556441,"t_decision_unix":1779712484.484498,"policy":"unified","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":512,"affinity_cache_ratio":0.40731901352426414,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":512,"estimated_new_tokens":745,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":113,"cache_hit":0,"new_prefill":1257,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":27387,"ongoing_decode_tokens":0,"pending_prefill_tokens":27387,"num_requests":1,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":1257,"score_linear":27387.0,"score_lmetric":28644},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":512,"new_prefill":745,"score_linear":-512.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":1257,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":1257,"score_linear":90488.0,"score_lmetric":91745},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":33,"cache_hit":0,"new_prefill":1257,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":1257,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":1257,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-512.0,"chosen_score_lmetric":0,"t_first_token":281619.525159946,"t_first_token_unix":1779712484.539103,"t_done":281619.860917975,"t_done_unix":1779712484.8748612},{"request_id":"1242838:3:1258161:53","session_id":"1242838","input_length":22262,"t_proxy_recv":281619.931133483,"t_decision_unix":1779712484.9450748,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":21504,"affinity_cache_ratio":0.9659509478034318,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":21504,"estimated_new_tokens":758,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":113,"cache_hit":21504,"new_prefill":758,"score_linear":-21504.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":27387,"ongoing_decode_tokens":0,"pending_prefill_tokens":27387,"num_requests":1,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":22262,"score_linear":27387.0,"score_lmetric":49649},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":22262,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":22262,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":22262,"score_linear":90488.0,"score_lmetric":112750},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":33,"cache_hit":0,"new_prefill":22262,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":22262,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":22262,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-21504.0,"chosen_score_lmetric":0,"t_first_token":281620.093681598,"t_first_token_unix":1779712485.1076252,"t_done":281620.644047035,"t_done_unix":1779712485.6579907},{"request_id":"1256273:3:1258499:55","session_id":"1256273","input_length":7934,"t_proxy_recv":281621.095317603,"t_decision_unix":1779712486.109259,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":6144,"affinity_cache_ratio":0.7743887068313587,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":6144,"estimated_new_tokens":1790,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":7934,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":0,"pending_prefill_tokens":45037,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":7934,"score_linear":80365.0,"score_lmetric":105942},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":7934,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":7934,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":7934,"score_linear":90488.0,"score_lmetric":98422},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":33,"cache_hit":6144,"new_prefill":1790,"score_linear":-6144.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":7934,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":7934,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-6144.0,"chosen_score_lmetric":0,"t_first_token":281621.289254749,"t_first_token_unix":1779712486.303198,"t_done":281621.899922852,"t_done_unix":1779712486.913866},{"request_id":"1258908:1:1258908:58","session_id":"1258908","input_length":9250,"t_proxy_recv":281622.547533974,"t_decision_unix":1779712487.5614755,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9250,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":9250,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":27387,"pending_prefill_tokens":17650,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":9250,"score_linear":80365.0,"score_lmetric":53800},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":9250,"score_linear":68131.0,"score_lmetric":77381},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":9250,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":9250,"score_linear":90488.0,"score_lmetric":99738},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":112793,"ongoing_decode_tokens":0,"pending_prefill_tokens":112793,"num_requests":1,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":9250,"score_linear":112793.0,"score_lmetric":122043},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":9250,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":9250,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281623.243108043,"t_first_token_unix":1779712488.257052,"t_done":281625.246378521,"t_done_unix":1779712490.2603219},{"request_id":"1259178:1:1259178:59","session_id":"1259178","input_length":14291,"t_proxy_recv":281623.509583566,"t_decision_unix":1779712488.5235252,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":14291,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":14291,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":27387,"pending_prefill_tokens":17650,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":14291,"score_linear":80365.0,"score_lmetric":63882},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":14291,"score_linear":68131.0,"score_lmetric":82422},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":9250,"ongoing_decode_tokens":9250,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":14291,"score_linear":9250.0,"score_lmetric":14291},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":14291,"score_linear":90488.0,"score_lmetric":104779},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":112793,"ongoing_decode_tokens":0,"pending_prefill_tokens":112793,"num_requests":1,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":14291,"score_linear":112793.0,"score_lmetric":127084},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":14291,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":14291,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281624.760403672,"t_first_token_unix":1779712489.7743473,"t_done":281626.716089211,"t_done_unix":1779712491.7300322},{"request_id":"1257525:1:1257525:49","session_id":"1257525","input_length":27387,"t_proxy_recv":281617.7312342,"t_decision_unix":1779712482.7451751,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":27387,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":27387,"score_linear":90488.0,"score_lmetric":117875},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":24,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281621.191940528,"t_first_token_unix":1779712486.2058842,"t_done":281626.816705022,"t_done_unix":1779712491.830648},{"request_id":"1253804:2:1258346:54","session_id":"1253804","input_length":52978,"t_proxy_recv":281620.649835558,"t_decision_unix":1779712485.663777,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":35328,"affinity_cache_ratio":0.6668428404243271,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35328,"estimated_new_tokens":17650,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":52978,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":27387,"ongoing_decode_tokens":0,"pending_prefill_tokens":27387,"num_requests":1,"active_p_offloads":0,"cached_blocks":165,"cache_hit":35328,"new_prefill":17650,"score_linear":-7941.0,"score_lmetric":45037},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":52978,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":3584,"new_prefill":49394,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":52978,"score_linear":90488.0,"score_lmetric":143466},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":33,"cache_hit":0,"new_prefill":52978,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":52978,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":52978,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7941.0,"chosen_score_lmetric":45037,"t_first_token":281626.449098421,"t_first_token_unix":1779712491.4630418,"t_done":281627.37110394,"t_done_unix":1779712492.3850472},{"request_id":"1260746:1:1260746:62","session_id":"1260746","input_length":9571,"t_proxy_recv":281629.054097334,"t_decision_unix":1779712494.0680385,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9571,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":9571,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":252,"cache_hit":0,"new_prefill":9571,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":9571,"score_linear":68131.0,"score_lmetric":77702},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":9571,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":9571,"score_linear":90488.0,"score_lmetric":100059},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":9571,"score_linear":125063.0,"score_lmetric":253908},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":9571,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":30655,"ongoing_decode_tokens":0,"pending_prefill_tokens":30655,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":9571,"score_linear":30655.0,"score_lmetric":40226}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281629.79759481,"t_first_token_unix":1779712494.8115385,"t_done":281630.190148836,"t_done_unix":1779712495.2040923},{"request_id":"1260792:1:1260792:63","session_id":"1260792","input_length":12522,"t_proxy_recv":281629.170094328,"t_decision_unix":1779712494.1840355,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":12522,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":9571,"ongoing_decode_tokens":0,"pending_prefill_tokens":9571,"num_requests":1,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":12522,"score_linear":9571.0,"score_lmetric":22093},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":252,"cache_hit":0,"new_prefill":12522,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":12522,"score_linear":68131.0,"score_lmetric":80653},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":12522,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":12522,"score_linear":90488.0,"score_lmetric":103010},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":12522,"score_linear":125063.0,"score_lmetric":259810},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":12522,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":30655,"ongoing_decode_tokens":0,"pending_prefill_tokens":30655,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":12522,"score_linear":30655.0,"score_lmetric":43177}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281630.201390808,"t_first_token_unix":1779712495.2153342,"t_done":281630.537848006,"t_done_unix":1779712495.5517912},{"request_id":"1261126:1:1261126:65","session_id":"1261126","input_length":5640,"t_proxy_recv":281630.207302841,"t_decision_unix":1779712495.2212443,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5640,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":132,"cache_hit":0,"new_prefill":5640,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55138,"ongoing_decode_tokens":0,"pending_prefill_tokens":2402,"num_requests":1,"active_p_offloads":0,"cached_blocks":252,"cache_hit":0,"new_prefill":5640,"score_linear":55138.0,"score_lmetric":8042},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":5640,"score_linear":68131.0,"score_lmetric":73771},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":12522,"ongoing_decode_tokens":12522,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":5640,"score_linear":12522.0,"score_lmetric":5640},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":5640,"score_linear":90488.0,"score_lmetric":96128},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":5640,"score_linear":125063.0,"score_lmetric":246046},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":5640,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":30655,"ongoing_decode_tokens":0,"pending_prefill_tokens":30655,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":5640,"score_linear":30655.0,"score_lmetric":36295}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281630.582796557,"t_first_token_unix":1779712495.5967398,"t_done":281630.750828965,"t_done_unix":1779712495.7647722},{"request_id":"1253804:3:1260997:64","session_id":"1253804","input_length":55138,"t_proxy_recv":281629.908197067,"t_decision_unix":1779712494.9221385,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9564365773151003,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":52736,"estimated_new_tokens":2402,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":9571,"ongoing_decode_tokens":9571,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":55138,"score_linear":9571.0,"score_lmetric":55138},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":252,"cache_hit":52736,"new_prefill":2402,"score_linear":-52736.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":55138,"score_linear":68131.0,"score_lmetric":123269},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":12522,"ongoing_decode_tokens":0,"pending_prefill_tokens":12522,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":3584,"new_prefill":51554,"score_linear":8938.0,"score_lmetric":64076},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":55138,"score_linear":90488.0,"score_lmetric":145626},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":55138,"score_linear":125063.0,"score_lmetric":345042},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":55138,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":30655,"ongoing_decode_tokens":0,"pending_prefill_tokens":30655,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":55138,"score_linear":30655.0,"score_lmetric":85793}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":281630.784953174,"t_first_token_unix":1779712495.7988966,"t_done":281631.36757545,"t_done_unix":1779712496.3815188},{"request_id":"1260327:1:1260327:61","session_id":"1260327","input_length":30655,"t_proxy_recv":281627.334239934,"t_decision_unix":1779712492.3481815,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":30655,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":30655,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":52978,"ongoing_decode_tokens":52978,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":218,"cache_hit":0,"new_prefill":30655,"score_linear":52978.0,"score_lmetric":30655},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":30655,"score_linear":68131.0,"score_lmetric":98786},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":30655,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":30655,"score_linear":90488.0,"score_lmetric":121143},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":30655,"score_linear":125063.0,"score_lmetric":296076},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":30655,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":30655,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281631.391686712,"t_first_token_unix":1779712496.4056299,"t_done":281632.036649475,"t_done_unix":1779712497.0505924},{"request_id":"1257525:2:1262205:67","session_id":"1257525","input_length":29988,"t_proxy_recv":281634.022155773,"t_decision_unix":1779712499.036097,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9048952914499133,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":27136,"estimated_new_tokens":2852,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":29988,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":256,"cache_hit":27136,"new_prefill":2852,"score_linear":-27136.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":29988,"score_linear":68131.0,"score_lmetric":98119},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":29988,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":29988,"score_linear":90488.0,"score_lmetric":120476},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":29988,"score_linear":125063.0,"score_lmetric":294742},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":29988,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":29988,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-27136.0,"chosen_score_lmetric":0,"t_first_token":281634.652866235,"t_first_token_unix":1779712499.6668098,"t_done":281634.679543197,"t_done_unix":1779712499.6934865},{"request_id":"1262757:1:1262757:73","session_id":"1262757","input_length":5465,"t_proxy_recv":281635.902128871,"t_decision_unix":1779712500.9160702,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":0,"pending_prefill_tokens":1205,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":5465,"score_linear":55989.0,"score_lmetric":6670},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":5465,"score_linear":68131.0,"score_lmetric":73596},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":0,"pending_prefill_tokens":15283,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":5465,"score_linear":15283.0,"score_lmetric":20748},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":5465,"score_linear":113053.0,"score_lmetric":196076},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":5465,"score_linear":125063.0,"score_lmetric":245696},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21510,"ongoing_decode_tokens":0,"pending_prefill_tokens":21510,"num_requests":1,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":5465,"score_linear":21510.0,"score_lmetric":26975}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281636.260875801,"t_first_token_unix":1779712501.274819,"t_done":281636.390521312,"t_done_unix":1779712501.4044645},{"request_id":"1261126:2:1262820:74","session_id":"1261126","input_length":6953,"t_proxy_recv":281636.073767298,"t_decision_unix":1779712501.0877087,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":5632,"affinity_cache_ratio":0.8100100675967209,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":5632,"estimated_new_tokens":1321,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":5632,"new_prefill":1321,"score_linear":-5632.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":0,"pending_prefill_tokens":1205,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":6953,"score_linear":55989.0,"score_lmetric":8158},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":6953,"score_linear":68131.0,"score_lmetric":75084},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":0,"pending_prefill_tokens":15283,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":6953,"score_linear":15283.0,"score_lmetric":22236},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":6953,"score_linear":113053.0,"score_lmetric":199052},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":6953,"score_linear":125063.0,"score_lmetric":248672},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":5465,"ongoing_decode_tokens":0,"pending_prefill_tokens":5465,"num_requests":1,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":6953,"score_linear":5465.0,"score_lmetric":12418},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21510,"ongoing_decode_tokens":0,"pending_prefill_tokens":21510,"num_requests":1,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":6953,"score_linear":21510.0,"score_lmetric":28463}],"chosen_score_linear":-5632.0,"chosen_score_lmetric":0,"t_first_token":281636.227660216,"t_first_token_unix":1779712501.2416039,"t_done":281636.465661732,"t_done_unix":1779712501.479605},{"request_id":"1262354:1:1262354:69","session_id":"1262354","input_length":21510,"t_proxy_recv":281634.620107813,"t_decision_unix":1779712499.6340492,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":21510,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":21510,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":29988,"ongoing_decode_tokens":0,"pending_prefill_tokens":2852,"num_requests":1,"active_p_offloads":0,"cached_blocks":256,"cache_hit":0,"new_prefill":21510,"score_linear":29988.0,"score_lmetric":24362},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":21510,"score_linear":68131.0,"score_lmetric":89641},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":21510,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":21510,"score_linear":113053.0,"score_lmetric":228166},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":21510,"score_linear":125063.0,"score_lmetric":277786},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":21510,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":21510,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281636.938803319,"t_first_token_unix":1779712501.9527469,"t_done":281637.544191266,"t_done_unix":1779712502.5581346},{"request_id":"1258611:1:1258611:56","session_id":"1258611","input_length":68131,"t_proxy_recv":281621.490193688,"t_decision_unix":1779712486.5041351,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":68131,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":68131,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":27387,"pending_prefill_tokens":17650,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":68131,"score_linear":80365.0,"score_lmetric":171562},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":68131,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":68131,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":68131,"score_linear":90488.0,"score_lmetric":158619},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":7934,"ongoing_decode_tokens":7934,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":33,"cache_hit":0,"new_prefill":68131,"score_linear":7934.0,"score_lmetric":68131},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":68131,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":68131,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281637.820263771,"t_first_token_unix":1779712502.834207,"t_done":281638.987966877,"t_done_unix":1779712504.00191},{"request_id":"1258611:2:1261594:66","session_id":"1258611","input_length":68320,"t_proxy_recv":281638.999314469,"t_decision_unix":1779712504.0132558,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":68096,"affinity_cache_ratio":0.9967213114754099,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":68096,"estimated_new_tokens":224,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":68320,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":68320,"score_linear":55989.0,"score_lmetric":68320},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":296,"cache_hit":68096,"new_prefill":224,"score_linear":-68096.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":68320,"score_linear":15283.0,"score_lmetric":68320},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":68320,"score_linear":113053.0,"score_lmetric":321786},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":68320,"score_linear":125063.0,"score_lmetric":371406},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":68320,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22109,"ongoing_decode_tokens":0,"pending_prefill_tokens":605,"num_requests":1,"active_p_offloads":0,"cached_blocks":128,"cache_hit":0,"new_prefill":68320,"score_linear":22109.0,"score_lmetric":68925}],"chosen_score_linear":-68096.0,"chosen_score_lmetric":0,"t_first_token":281639.144743617,"t_first_token_unix":1779712504.1586869,"t_done":281639.184961742,"t_done_unix":1779712504.1989057},{"request_id":"1258611:3:1262709:71","session_id":"1258611","input_length":68377,"t_proxy_recv":281639.202831228,"t_decision_unix":1779712504.2167726,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":68096,"affinity_cache_ratio":0.9958904309928778,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":68096,"estimated_new_tokens":281,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5990,"ongoing_decode_tokens":0,"pending_prefill_tokens":5990,"num_requests":1,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":68377,"score_linear":5990.0,"score_lmetric":74367},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":68377,"score_linear":55989.0,"score_lmetric":68377},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":296,"cache_hit":68096,"new_prefill":281,"score_linear":-68096.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":68377,"score_linear":15283.0,"score_lmetric":68377},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":68377,"score_linear":113053.0,"score_lmetric":321900},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":68377,"score_linear":125063.0,"score_lmetric":371520},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":68377,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22109,"ongoing_decode_tokens":22109,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":128,"cache_hit":0,"new_prefill":68377,"score_linear":22109.0,"score_lmetric":68377}],"chosen_score_linear":-68096.0,"chosen_score_lmetric":0,"t_first_token":281639.348735,"t_first_token_unix":1779712504.362679,"t_done":281639.779690699,"t_done_unix":1779712504.793634},{"request_id":"1263611:1:1263611:76","session_id":"1263611","input_length":5990,"t_proxy_recv":281639.095986419,"t_decision_unix":1779712504.1099277,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5990,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":5990,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":5990,"score_linear":55989.0,"score_lmetric":5990},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68320,"ongoing_decode_tokens":0,"pending_prefill_tokens":224,"num_requests":1,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":5990,"score_linear":68320.0,"score_lmetric":6214},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":5990,"score_linear":15283.0,"score_lmetric":5990},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":5990,"score_linear":113053.0,"score_lmetric":197126},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":5990,"score_linear":125063.0,"score_lmetric":246746},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":5990,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22109,"ongoing_decode_tokens":0,"pending_prefill_tokens":605,"num_requests":1,"active_p_offloads":0,"cached_blocks":128,"cache_hit":0,"new_prefill":5990,"score_linear":22109.0,"score_lmetric":6595}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281639.487650025,"t_first_token_unix":1779712504.5015934,"t_done":281639.879946014,"t_done_unix":1779712504.893889},{"request_id":"1262354:2:1263593:75","session_id":"1262354","input_length":22109,"t_proxy_recv":281638.983068796,"t_decision_unix":1779712503.99701,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":21504,"affinity_cache_ratio":0.9726355782712922,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":21504,"estimated_new_tokens":605,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":22109,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":22109,"score_linear":55989.0,"score_lmetric":22109},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":68131,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":22109,"score_linear":68131.0,"score_lmetric":22109},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":22109,"score_linear":15283.0,"score_lmetric":22109},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":22109,"score_linear":113053.0,"score_lmetric":229364},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":22109,"score_linear":125063.0,"score_lmetric":278984},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":22109,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":128,"cache_hit":21504,"new_prefill":605,"score_linear":-21504.0,"score_lmetric":0}],"chosen_score_linear":-21504.0,"chosen_score_lmetric":0,"t_first_token":281639.141758944,"t_first_token_unix":1779712504.1557026,"t_done":281640.373531773,"t_done_unix":1779712505.387475},{"request_id":"1263971:1:1263971:80","session_id":"1263971","input_length":6336,"t_proxy_recv":281640.404208955,"t_decision_unix":1779712505.4181504,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":6336,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":6336,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":6336,"score_linear":55989.0,"score_lmetric":6336},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":6336,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":6336,"score_linear":71575.0,"score_lmetric":125256},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":6336,"score_linear":113053.0,"score_lmetric":197818},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":6336,"score_linear":125063.0,"score_lmetric":247438},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":6336,"score_linear":50783.0,"score_lmetric":57119},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":6336,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281640.821832732,"t_first_token_unix":1779712505.8357763,"t_done":281641.186596209,"t_done_unix":1779712506.20054},{"request_id":"1257323:1:1257323:48","session_id":"1257323","input_length":90488,"t_proxy_recv":281617.130594217,"t_decision_unix":1779712482.1445353,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":90488,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":24,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281644.811339585,"t_first_token_unix":1779712509.825283,"t_done":281645.227142409,"t_done_unix":1779712510.2410858},{"request_id":"1239034:3:1262346:68","session_id":"1239034","input_length":22565,"t_proxy_recv":281634.586759724,"t_decision_unix":1779712499.6007009,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":20480,"affinity_cache_ratio":0.9076002658985154,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":20480,"estimated_new_tokens":2085,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":22565,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":29988,"ongoing_decode_tokens":0,"pending_prefill_tokens":2852,"num_requests":1,"active_p_offloads":0,"cached_blocks":256,"cache_hit":0,"new_prefill":22565,"score_linear":29988.0,"score_lmetric":25417},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":22565,"score_linear":68131.0,"score_lmetric":90696},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":22565,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":20480,"new_prefill":2085,"score_linear":70008.0,"score_lmetric":92573},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":22565,"score_linear":125063.0,"score_lmetric":279896},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":22565,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":22565,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":70008.0,"chosen_score_lmetric":92573,"t_first_token":281644.813138464,"t_first_token_unix":1779712509.827082,"t_done":281645.353453759,"t_done_unix":1779712510.3673968},{"request_id":"1265000:1:1265000:83","session_id":"1265000","input_length":301,"t_proxy_recv":281644.030428555,"t_decision_unix":1779712509.0443697,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":301,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":301,"score_linear":110048.0,"score_lmetric":110349},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":301,"score_linear":55989.0,"score_lmetric":301},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":301,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":301,"score_linear":71575.0,"score_lmetric":113186},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":301,"score_linear":113053.0,"score_lmetric":185748},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":301,"score_linear":125063.0,"score_lmetric":235368},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":301,"score_linear":50783.0,"score_lmetric":51084},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":301,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281644.068473301,"t_first_token_unix":1779712509.0824165,"t_done":281645.583792093,"t_done_unix":1779712510.5977352},{"request_id":"1265128:1:1265128:85","session_id":"1265128","input_length":14028,"t_proxy_recv":281644.364386238,"t_decision_unix":1779712509.3783274,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":14028,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":14028,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":14028,"score_linear":110048.0,"score_lmetric":124076},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":14028,"score_linear":55989.0,"score_lmetric":14028},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":301,"ongoing_decode_tokens":301,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":14028,"score_linear":301.0,"score_lmetric":14028},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":14028,"score_linear":71575.0,"score_lmetric":140640},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":14028,"score_linear":113053.0,"score_lmetric":213202},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":14028,"score_linear":125063.0,"score_lmetric":262822},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":14028,"score_linear":50783.0,"score_lmetric":64811},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":23540,"ongoing_decode_tokens":0,"pending_prefill_tokens":23540,"num_requests":1,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":14028,"score_linear":23540.0,"score_lmetric":37568}],"chosen_score_linear":301.0,"chosen_score_lmetric":14028,"t_first_token":281645.573726971,"t_first_token_unix":1779712510.5876703,"t_done":281646.008596605,"t_done_unix":1779712511.02254},{"request_id":"1266067:1:1266067:91","session_id":"1266067","input_length":579,"t_proxy_recv":281647.581146634,"t_decision_unix":1779712512.5950882,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":579,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":579,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":579,"score_linear":135235.0,"score_lmetric":227596},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":60059,"ongoing_decode_tokens":55989,"pending_prefill_tokens":4070,"num_requests":2,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":579,"score_linear":60059.0,"score_lmetric":9298},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13178,"ongoing_decode_tokens":0,"pending_prefill_tokens":13178,"num_requests":1,"active_p_offloads":0,"cached_blocks":323,"cache_hit":0,"new_prefill":579,"score_linear":13178.0,"score_lmetric":13757},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":92560,"ongoing_decode_tokens":15283,"pending_prefill_tokens":64989,"num_requests":3,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":579,"score_linear":92560.0,"score_lmetric":196704},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":258,"cache_hit":0,"new_prefill":579,"score_linear":27639.0,"score_lmetric":579},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":579,"score_linear":125063.0,"score_lmetric":235924},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":579,"score_linear":50783.0,"score_lmetric":51362},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":45884,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":579,"score_linear":45884.0,"score_lmetric":1158}],"chosen_score_linear":27639.0,"chosen_score_lmetric":579,"t_first_token":281647.69632674,"t_first_token_unix":1779712512.7102702,"t_done":281647.863802933,"t_done_unix":1779712512.877746},{"request_id":"1263611:2:1265770:88","session_id":"1263611","input_length":13178,"t_proxy_recv":281646.681412816,"t_decision_unix":1779712511.6953537,"policy":"unified","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":2,"affinity_cache_hit":5632,"affinity_cache_ratio":0.4273789649415693,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":13178,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":5632,"new_prefill":7546,"score_linear":129603.0,"score_lmetric":241530},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":13178,"score_linear":55989.0,"score_lmetric":13178},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":323,"cache_hit":0,"new_prefill":13178,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":13178,"score_linear":71575.0,"score_lmetric":138940},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":258,"cache_hit":0,"new_prefill":13178,"score_linear":27639.0,"score_lmetric":13178},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":13178,"score_linear":125063.0,"score_lmetric":261122},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":13178,"score_linear":50783.0,"score_lmetric":63961},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":0,"pending_prefill_tokens":23868,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":13178,"score_linear":45884.0,"score_lmetric":74092}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281647.784818503,"t_first_token_unix":1779712512.7987628,"t_done":281648.036919035,"t_done_unix":1779712513.0508623},{"request_id":"1250772:3:1265019:84","session_id":"1250772","input_length":23540,"t_proxy_recv":281644.102677369,"t_decision_unix":1779712509.1166184,"policy":"unified","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":7,"affinity_cache_hit":10752,"affinity_cache_ratio":0.45675446049277824,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":23540,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":23540,"score_linear":110048.0,"score_lmetric":133588},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":23540,"score_linear":55989.0,"score_lmetric":23540},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":301,"ongoing_decode_tokens":301,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":23540,"score_linear":301.0,"score_lmetric":23540},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":23540,"score_linear":71575.0,"score_lmetric":159664},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":23540,"score_linear":113053.0,"score_lmetric":232226},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":10752,"new_prefill":12788,"score_linear":114311.0,"score_lmetric":260342},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":23540,"score_linear":50783.0,"score_lmetric":74323},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":23540,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281646.7803376,"t_first_token_unix":1779712511.794281,"t_done":281648.40654568,"t_done_unix":1779712513.420494},{"request_id":"1265942:1:1265942:89","session_id":"1265942","input_length":4070,"t_proxy_recv":281647.305981195,"t_decision_unix":1779712512.3199224,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":4070,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4070,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":4070,"score_linear":135235.0,"score_lmetric":234578},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":4070,"score_linear":55989.0,"score_lmetric":4070},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13178,"ongoing_decode_tokens":0,"pending_prefill_tokens":13178,"num_requests":1,"active_p_offloads":0,"cached_blocks":323,"cache_hit":0,"new_prefill":4070,"score_linear":13178.0,"score_lmetric":17248},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":4070,"score_linear":71575.0,"score_lmetric":120724},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":258,"cache_hit":0,"new_prefill":4070,"score_linear":27639.0,"score_lmetric":4070},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":4070,"score_linear":125063.0,"score_lmetric":242906},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":4070,"score_linear":50783.0,"score_lmetric":54853},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":45884,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":4070,"score_linear":45884.0,"score_lmetric":8140}],"chosen_score_linear":55989.0,"chosen_score_lmetric":4070,"t_first_token":281647.626306781,"t_first_token_unix":1779712512.64025,"t_done":281648.476470111,"t_done_unix":1779712513.490415},{"request_id":"1262354:3:1265178:86","session_id":"1262354","input_length":22344,"t_proxy_recv":281644.46615866,"t_decision_unix":1779712509.4801002,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":22016,"affinity_cache_ratio":0.9853204439670605,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":22016,"estimated_new_tokens":328,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":22344,"score_linear":110048.0,"score_lmetric":132392},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":22344,"score_linear":55989.0,"score_lmetric":22344},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14329,"ongoing_decode_tokens":301,"pending_prefill_tokens":14028,"num_requests":2,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":22344,"score_linear":14329.0,"score_lmetric":72744},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":22344,"score_linear":71575.0,"score_lmetric":157272},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":22344,"score_linear":113053.0,"score_lmetric":229834},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":22344,"score_linear":125063.0,"score_lmetric":279454},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":22344,"score_linear":50783.0,"score_lmetric":73127},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":23540,"ongoing_decode_tokens":0,"pending_prefill_tokens":23540,"num_requests":1,"active_p_offloads":0,"cached_blocks":141,"cache_hit":22016,"new_prefill":328,"score_linear":1524.0,"score_lmetric":23868}],"chosen_score_linear":1524.0,"chosen_score_lmetric":23868,"t_first_token":281646.782396148,"t_first_token_unix":1779712511.7963393,"t_done":281649.201234603,"t_done_unix":1779712514.2151778},{"request_id":"1266440:1:1266440:92","session_id":"1266440","input_length":5381,"t_proxy_recv":281648.653880804,"t_decision_unix":1779712513.6678221,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":5381,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":5381,"score_linear":135235.0,"score_lmetric":237200},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":269,"cache_hit":0,"new_prefill":5381,"score_linear":55989.0,"score_lmetric":5381},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":348,"cache_hit":0,"new_prefill":5381,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":92560,"ongoing_decode_tokens":15283,"pending_prefill_tokens":64989,"num_requests":3,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":5381,"score_linear":92560.0,"score_lmetric":211110},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":5381,"score_linear":27639.0,"score_lmetric":5381},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":5381,"score_linear":125063.0,"score_lmetric":245528},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":5381,"score_linear":50783.0,"score_lmetric":56164},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22344,"ongoing_decode_tokens":22344,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":187,"cache_hit":0,"new_prefill":5381,"score_linear":22344.0,"score_lmetric":5381}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281649.031256184,"t_first_token_unix":1779712514.0451994,"t_done":281649.335159306,"t_done_unix":1779712514.3491025},{"request_id":"1263727:1:1263727:77","session_id":"1263727","input_length":50783,"t_proxy_recv":281639.567225064,"t_decision_unix":1779712504.5811663,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":50783,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5990,"ongoing_decode_tokens":5990,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":50783,"score_linear":5990.0,"score_lmetric":50783},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":50783,"score_linear":55989.0,"score_lmetric":50783},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68377,"ongoing_decode_tokens":68377,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":50783,"score_linear":68377.0,"score_lmetric":50783},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":50783,"score_linear":15283.0,"score_lmetric":50783},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":50783,"score_linear":113053.0,"score_lmetric":286712},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":50783,"score_linear":125063.0,"score_lmetric":336332},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":50783,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22109,"ongoing_decode_tokens":22109,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":128,"cache_hit":0,"new_prefill":50783,"score_linear":22109.0,"score_lmetric":50783}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281649.303857799,"t_first_token_unix":1779712514.317801,"t_done":281649.478631853,"t_done_unix":1779712514.492575},{"request_id":"1253804:4:1262690:70","session_id":"1253804","input_length":55989,"t_proxy_recv":281635.717896622,"t_decision_unix":1779712500.731838,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":54784,"affinity_cache_ratio":0.978477915304792,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54784,"estimated_new_tokens":1205,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":55989,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":261,"cache_hit":54784,"new_prefill":1205,"score_linear":-54784.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":55989,"score_linear":68131.0,"score_lmetric":124120},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":231,"cache_hit":3584,"new_prefill":52405,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":55989,"score_linear":113053.0,"score_lmetric":297124},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":55989,"score_linear":125063.0,"score_lmetric":346744},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":55989,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21510,"ongoing_decode_tokens":0,"pending_prefill_tokens":21510,"num_requests":1,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":55989,"score_linear":21510.0,"score_lmetric":77499}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":281636.142958629,"t_first_token_unix":1779712501.1569018,"t_done":281650.022917276,"t_done_unix":1779712515.0368607},{"request_id":"1263971:2:1266797:94","session_id":"1263971","input_length":9504,"t_proxy_recv":281650.002890751,"t_decision_unix":1779712515.016832,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":6144,"affinity_cache_ratio":0.6464646464646465,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":6144,"estimated_new_tokens":3360,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":9504,"score_linear":135235.0,"score_lmetric":245446},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":269,"cache_hit":0,"new_prefill":9504,"score_linear":55989.0,"score_lmetric":9504},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":9504,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":92560,"ongoing_decode_tokens":15283,"pending_prefill_tokens":64989,"num_requests":3,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":9504,"score_linear":92560.0,"score_lmetric":223479},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":9504,"score_linear":27639.0,"score_lmetric":9504},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":9504,"score_linear":125063.0,"score_lmetric":253774},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":17466,"ongoing_decode_tokens":0,"pending_prefill_tokens":17466,"num_requests":1,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":9504,"score_linear":17466.0,"score_lmetric":26970},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":188,"cache_hit":6144,"new_prefill":3360,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-6144.0,"chosen_score_lmetric":0,"t_first_token":281650.363186733,"t_first_token_unix":1779712515.3771303,"t_done":281650.51585987,"t_done_unix":1779712515.529803},{"request_id":"1260792:2:1265951:90","session_id":"1260792","input_length":20985,"t_proxy_recv":281647.334183479,"t_decision_unix":1779712512.348125,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":12288,"affinity_cache_ratio":0.5855611150822015,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":12288,"estimated_new_tokens":8697,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":20985,"score_linear":135235.0,"score_lmetric":268408},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":60059,"ongoing_decode_tokens":55989,"pending_prefill_tokens":4070,"num_requests":2,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":20985,"score_linear":60059.0,"score_lmetric":50110},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13178,"ongoing_decode_tokens":0,"pending_prefill_tokens":13178,"num_requests":1,"active_p_offloads":0,"cached_blocks":323,"cache_hit":0,"new_prefill":20985,"score_linear":13178.0,"score_lmetric":34163},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":12288,"new_prefill":8697,"score_linear":59287.0,"score_lmetric":129978},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":258,"cache_hit":0,"new_prefill":20985,"score_linear":27639.0,"score_lmetric":20985},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":20985,"score_linear":125063.0,"score_lmetric":276736},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":20985,"score_linear":50783.0,"score_lmetric":71768},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":45884,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":20985,"score_linear":45884.0,"score_lmetric":41970}],"chosen_score_linear":59287.0,"chosen_score_lmetric":129978,"t_first_token":281652.263416458,"t_first_token_unix":1779712517.2773604,"t_done":281652.779359782,"t_done_unix":1779712517.793303},{"request_id":"1263749:1:1263749:79","session_id":"1263749","input_length":56292,"t_proxy_recv":281639.639178507,"t_decision_unix":1779712504.65312,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":56292,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":56292,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5990,"ongoing_decode_tokens":5990,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":56292,"score_linear":5990.0,"score_lmetric":56292},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":56292,"score_linear":55989.0,"score_lmetric":56292},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68377,"ongoing_decode_tokens":68377,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":56292,"score_linear":68377.0,"score_lmetric":56292},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":56292,"score_linear":15283.0,"score_lmetric":56292},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":56292,"score_linear":113053.0,"score_lmetric":297730},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":56292,"score_linear":125063.0,"score_lmetric":347350},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":56292,"score_linear":50783.0,"score_lmetric":107075},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22109,"ongoing_decode_tokens":22109,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":128,"cache_hit":0,"new_prefill":56292,"score_linear":22109.0,"score_lmetric":56292}],"chosen_score_linear":15283.0,"chosen_score_lmetric":56292,"t_first_token":281651.094674896,"t_first_token_unix":1779712516.1086185,"t_done":281653.725721157,"t_done_unix":1779712518.7396643},{"request_id":"1266668:1:1266668:93","session_id":"1266668","input_length":17466,"t_proxy_recv":281649.55493365,"t_decision_unix":1779712514.5688753,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":17466,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":13370,"score_linear":131139.0,"score_lmetric":253178},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":269,"cache_hit":0,"new_prefill":17466,"score_linear":55989.0,"score_lmetric":17466},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":17466,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":92560,"ongoing_decode_tokens":15283,"pending_prefill_tokens":64989,"num_requests":3,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":17466,"score_linear":92560.0,"score_lmetric":247365},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":17466,"score_linear":27639.0,"score_lmetric":17466},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":17466,"score_linear":125063.0,"score_lmetric":269698},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":17466,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":188,"cache_hit":0,"new_prefill":17466,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281651.268629788,"t_first_token_unix":1779712516.2825732,"t_done":281655.184330136,"t_done_unix":1779712520.1982732},{"request_id":"1265942:2:1268069:97","session_id":"1265942","input_length":10457,"t_proxy_recv":281654.311613499,"t_decision_unix":1779712519.3255548,"policy":"unified","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":4096,"affinity_cache_ratio":0.39169934015492014,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":4096,"estimated_new_tokens":6361,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":10457,"score_linear":135235.0,"score_lmetric":247352},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":273,"cache_hit":4096,"new_prefill":6361,"score_linear":-4096.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":10457,"score_linear":126924.0,"score_lmetric":118437},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":10457,"score_linear":15283.0,"score_lmetric":10457},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":10457,"score_linear":27639.0,"score_lmetric":10457},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":10457,"score_linear":125063.0,"score_lmetric":255680},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":17466,"ongoing_decode_tokens":17466,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":10457,"score_linear":17466.0,"score_lmetric":10457},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28671,"ongoing_decode_tokens":0,"pending_prefill_tokens":6143,"num_requests":1,"active_p_offloads":0,"cached_blocks":194,"cache_hit":0,"new_prefill":10457,"score_linear":28671.0,"score_lmetric":16600}],"chosen_score_linear":-4096.0,"chosen_score_lmetric":0,"t_first_token":281654.88806046,"t_first_token_unix":1779712519.9020035,"t_done":281655.24791046,"t_done_unix":1779712520.2618535},{"request_id":"1262354:4:1267821:95","session_id":"1262354","input_length":28671,"t_proxy_recv":281653.611647284,"t_decision_unix":1779712518.6255884,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":22528,"affinity_cache_ratio":0.7857416902096195,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":22528,"estimated_new_tokens":6143,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":28671,"score_linear":135235.0,"score_lmetric":283780},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":273,"cache_hit":0,"new_prefill":28671,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28671,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":71575,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":248,"cache_hit":0,"new_prefill":28671,"score_linear":71575.0,"score_lmetric":57342},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":28671,"score_linear":27639.0,"score_lmetric":28671},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":28671,"score_linear":125063.0,"score_lmetric":292108},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":17466,"ongoing_decode_tokens":17466,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":28671,"score_linear":17466.0,"score_lmetric":28671},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":194,"cache_hit":22528,"new_prefill":6143,"score_linear":-22528.0,"score_lmetric":0}],"chosen_score_linear":-22528.0,"chosen_score_lmetric":0,"t_first_token":281654.798565432,"t_first_token_unix":1779712519.8125093,"t_done":281656.107109838,"t_done_unix":1779712521.121053},{"request_id":"1268634:1:1268634:99","session_id":"1268634","input_length":2298,"t_proxy_recv":281656.55507369,"t_decision_unix":1779712521.5690155,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":2298,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":2298,"score_linear":135235.0,"score_lmetric":231034},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":2298,"score_linear":102074.0,"score_lmetric":100788},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":2298,"score_linear":126924.0,"score_lmetric":110278},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":2298,"score_linear":15283.0,"score_lmetric":2298},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":2298,"score_linear":27639.0,"score_lmetric":2298},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":2298,"score_linear":125063.0,"score_lmetric":239362},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":341,"cache_hit":0,"new_prefill":2298,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":206,"cache_hit":0,"new_prefill":2298,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281656.767557353,"t_first_token_unix":1779712521.7815008,"t_done":281656.882229557,"t_done_unix":1779712521.8961728},{"request_id":"1268649:1:1268649:100","session_id":"1268649","input_length":18638,"t_proxy_recv":281656.56473432,"t_decision_unix":1779712521.578676,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":18638,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":18638,"score_linear":135235.0,"score_lmetric":263714},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":18638,"score_linear":102074.0,"score_lmetric":117128},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":18638,"score_linear":126924.0,"score_lmetric":126618},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":18638,"score_linear":15283.0,"score_lmetric":18638},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":18638,"score_linear":27639.0,"score_lmetric":18638},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":18638,"score_linear":125063.0,"score_lmetric":272042},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":2298,"ongoing_decode_tokens":0,"pending_prefill_tokens":2298,"num_requests":1,"active_p_offloads":0,"cached_blocks":341,"cache_hit":0,"new_prefill":18638,"score_linear":2298.0,"score_lmetric":20936},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":206,"cache_hit":0,"new_prefill":18638,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281658.504652911,"t_first_token_unix":1779712523.5185964,"t_done":281659.321850387,"t_done_unix":1779712524.3357937},{"request_id":"1263971:3:1269088:104","session_id":"1263971","input_length":13955,"t_proxy_recv":281657.946806063,"t_decision_unix":1779712522.9607475,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":9216,"affinity_cache_ratio":0.660408455750627,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":9216,"estimated_new_tokens":4739,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":13955,"score_linear":135235.0,"score_lmetric":254348},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":13955,"score_linear":102074.0,"score_lmetric":112445},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":13955,"score_linear":126924.0,"score_lmetric":121935},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":13955,"score_linear":15283.0,"score_lmetric":13955},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":39959,"ongoing_decode_tokens":27639,"pending_prefill_tokens":12320,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":13955,"score_linear":39959.0,"score_lmetric":52550},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":13955,"score_linear":125063.0,"score_lmetric":262676},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12195,"ongoing_decode_tokens":0,"pending_prefill_tokens":12195,"num_requests":1,"active_p_offloads":0,"cached_blocks":345,"cache_hit":0,"new_prefill":13955,"score_linear":12195.0,"score_lmetric":26150},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":42667,"ongoing_decode_tokens":0,"pending_prefill_tokens":19115,"num_requests":2,"active_p_offloads":0,"cached_blocks":206,"cache_hit":9216,"new_prefill":4739,"score_linear":33451.0,"score_lmetric":47708}],"chosen_score_linear":33451.0,"chosen_score_lmetric":47708,"t_first_token":281659.038698998,"t_first_token_unix":1779712524.0526423,"t_done":281659.373407694,"t_done_unix":1779712524.387351},{"request_id":"1262354:5:1269336:105","session_id":"1262354","input_length":28906,"t_proxy_recv":281658.900673763,"t_decision_unix":1779712523.9146147,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":28672,"affinity_cache_ratio":0.9919047948522798,"affinity_num_requests":3,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":28672,"estimated_new_tokens":234,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":28906,"score_linear":135235.0,"score_lmetric":284250},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":28906,"score_linear":102074.0,"score_lmetric":127396},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28906,"score_linear":126924.0,"score_lmetric":136886},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28906,"score_linear":15283.0,"score_lmetric":28906},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":39959,"ongoing_decode_tokens":39959,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":28906,"score_linear":39959.0,"score_lmetric":57812},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":28906,"score_linear":125063.0,"score_lmetric":292578},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12195,"ongoing_decode_tokens":12195,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":345,"cache_hit":0,"new_prefill":28906,"score_linear":12195.0,"score_lmetric":28906},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":56622,"ongoing_decode_tokens":18638,"pending_prefill_tokens":5216,"num_requests":3,"active_p_offloads":0,"cached_blocks":206,"cache_hit":28672,"new_prefill":234,"score_linear":27950.0,"score_lmetric":16350}],"chosen_score_linear":27950.0,"chosen_score_lmetric":16350,"t_first_token":281659.075213701,"t_first_token_unix":1779712524.0891578,"t_done":281659.998859456,"t_done_unix":1779712525.012803},{"request_id":"1250772:4:1268869:103","session_id":"1250772","input_length":24029,"t_proxy_recv":281657.203261602,"t_decision_unix":1779712522.2172031,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":23552,"affinity_cache_ratio":0.9801489866411419,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":23552,"estimated_new_tokens":477,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":24029,"score_linear":135235.0,"score_lmetric":274496},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":24029,"score_linear":102074.0,"score_lmetric":122519},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":24029,"score_linear":126924.0,"score_lmetric":132009},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":24029,"score_linear":15283.0,"score_lmetric":24029},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":39959,"ongoing_decode_tokens":27639,"pending_prefill_tokens":12320,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":24029,"score_linear":39959.0,"score_lmetric":72698},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":10752,"new_prefill":13277,"score_linear":114311.0,"score_lmetric":261320},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12195,"ongoing_decode_tokens":0,"pending_prefill_tokens":12195,"num_requests":1,"active_p_offloads":0,"cached_blocks":345,"cache_hit":0,"new_prefill":24029,"score_linear":12195.0,"score_lmetric":36224},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":18638,"ongoing_decode_tokens":0,"pending_prefill_tokens":18638,"num_requests":1,"active_p_offloads":0,"cached_blocks":206,"cache_hit":23552,"new_prefill":477,"score_linear":-4914.0,"score_lmetric":19115}],"chosen_score_linear":-4914.0,"chosen_score_lmetric":19115,"t_first_token":281659.035549414,"t_first_token_unix":1779712524.0494926,"t_done":281660.150123135,"t_done_unix":1779712525.1640663},{"request_id":"1268831:1:1268831:101","session_id":"1268831","input_length":12195,"t_proxy_recv":281657.047487577,"t_decision_unix":1779712522.061429,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":12195,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":12195,"score_linear":135235.0,"score_lmetric":250828},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":12195,"score_linear":102074.0,"score_lmetric":110685},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":12195,"score_linear":126924.0,"score_lmetric":120175},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":12195,"score_linear":15283.0,"score_lmetric":12195},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":12195,"score_linear":27639.0,"score_lmetric":12195},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":12195,"score_linear":125063.0,"score_lmetric":259156},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":345,"cache_hit":0,"new_prefill":12195,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":18638,"ongoing_decode_tokens":0,"pending_prefill_tokens":18638,"num_requests":1,"active_p_offloads":0,"cached_blocks":206,"cache_hit":0,"new_prefill":12195,"score_linear":18638.0,"score_lmetric":30833}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281658.05433334,"t_first_token_unix":1779712523.0682764,"t_done":281660.232578095,"t_done_unix":1779712525.2465212},{"request_id":"1269373:1:1269373:106","session_id":"1269373","input_length":14104,"t_proxy_recv":281659.024321864,"t_decision_unix":1779712524.0382628,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":6424,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":6424,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":10008,"score_linear":131139.0,"score_lmetric":246454},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":14104,"score_linear":102074.0,"score_lmetric":112594},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":14104,"score_linear":126924.0,"score_lmetric":122084},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":14104,"score_linear":15283.0,"score_lmetric":14104},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":39959,"ongoing_decode_tokens":39959,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":14104,"score_linear":39959.0,"score_lmetric":28208},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":14104,"score_linear":125063.0,"score_lmetric":262974},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12195,"ongoing_decode_tokens":12195,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":345,"cache_hit":7680,"new_prefill":6424,"score_linear":4515.0,"score_lmetric":6424},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":85528,"ongoing_decode_tokens":18638,"pending_prefill_tokens":5450,"num_requests":4,"active_p_offloads":0,"cached_blocks":206,"cache_hit":0,"new_prefill":14104,"score_linear":85528.0,"score_lmetric":78216}],"chosen_score_linear":4515.0,"chosen_score_lmetric":6424,"t_first_token":281659.754458,"t_first_token_unix":1779712524.7684016,"t_done":281661.183397467,"t_done_unix":1779712526.1973407},{"request_id":"1268861:1:1268861:102","session_id":"1268861","input_length":12320,"t_proxy_recv":281657.165561066,"t_decision_unix":1779712522.1795025,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":12320,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":12320,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":12320,"score_linear":135235.0,"score_lmetric":251078},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":12320,"score_linear":102074.0,"score_lmetric":110810},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":12320,"score_linear":126924.0,"score_lmetric":120300},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":12320,"score_linear":15283.0,"score_lmetric":12320},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":12320,"score_linear":27639.0,"score_lmetric":12320},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":12320,"score_linear":125063.0,"score_lmetric":259406},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12195,"ongoing_decode_tokens":0,"pending_prefill_tokens":12195,"num_requests":1,"active_p_offloads":0,"cached_blocks":345,"cache_hit":0,"new_prefill":12320,"score_linear":12195.0,"score_lmetric":24515},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":18638,"ongoing_decode_tokens":0,"pending_prefill_tokens":18638,"num_requests":1,"active_p_offloads":0,"cached_blocks":206,"cache_hit":0,"new_prefill":12320,"score_linear":18638.0,"score_lmetric":30958}],"chosen_score_linear":27639.0,"chosen_score_lmetric":12320,"t_first_token":281658.18794742,"t_first_token_unix":1779712523.2018912,"t_done":281661.191245104,"t_done_unix":1779712526.2051883},{"request_id":"1269901:1:1269901:107","session_id":"1269901","input_length":1323,"t_proxy_recv":281660.582682963,"t_decision_unix":1779712525.5966232,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":1323,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":1323,"score_linear":135235.0,"score_lmetric":229084},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":1323,"score_linear":102074.0,"score_lmetric":99813},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":1323,"score_linear":126924.0,"score_lmetric":109303},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":1323,"score_linear":15283.0,"score_lmetric":1323},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":39959,"ongoing_decode_tokens":39959,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":1323,"score_linear":39959.0,"score_lmetric":2646},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":1323,"score_linear":125063.0,"score_lmetric":237412},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":14104,"ongoing_decode_tokens":14104,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":369,"cache_hit":0,"new_prefill":1323,"score_linear":14104.0,"score_lmetric":1323},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":252,"cache_hit":0,"new_prefill":1323,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281660.731479365,"t_first_token_unix":1779712525.7454224,"t_done":281661.339896195,"t_done_unix":1779712526.3538394},{"request_id":"1260327:2:1270005:108","session_id":"1260327","input_length":30828,"t_proxy_recv":281661.008416828,"t_decision_unix":1779712526.0223575,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":30720,"affinity_cache_ratio":0.9964966913195796,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":30720,"estimated_new_tokens":108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":30828,"score_linear":135235.0,"score_lmetric":288094},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":30828,"score_linear":102074.0,"score_lmetric":129318},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":30828,"score_linear":126924.0,"score_lmetric":138808},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":30828,"score_linear":15283.0,"score_lmetric":30828},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":39959,"ongoing_decode_tokens":39959,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":30828,"score_linear":39959.0,"score_lmetric":61656},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":30828,"score_linear":125063.0,"score_lmetric":296422},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":14104,"ongoing_decode_tokens":14104,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":369,"cache_hit":0,"new_prefill":30828,"score_linear":14104.0,"score_lmetric":30828},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":1323,"ongoing_decode_tokens":1323,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":252,"cache_hit":30720,"new_prefill":108,"score_linear":-29397.0,"score_lmetric":108}],"chosen_score_linear":-29397.0,"chosen_score_lmetric":108,"t_first_token":281661.096482034,"t_first_token_unix":1779712526.1104257,"t_done":281661.481661983,"t_done_unix":1779712526.4956071},{"request_id":"1266440:2:1270501:112","session_id":"1266440","input_length":11128,"t_proxy_recv":281662.777480702,"t_decision_unix":1779712527.791422,"policy":"unified","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":3,"affinity_cache_hit":5120,"affinity_cache_ratio":0.46010064701653486,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":11128,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":11128,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":11128,"score_linear":135235.0,"score_lmetric":248694},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":11128,"score_linear":102074.0,"score_lmetric":109618},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":5120,"new_prefill":6008,"score_linear":121804.0,"score_lmetric":113988},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":11128,"score_linear":15283.0,"score_lmetric":11128},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":11128,"score_linear":27639.0,"score_lmetric":11128},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":11128,"score_linear":125063.0,"score_lmetric":257022},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":57803,"ongoing_decode_tokens":0,"pending_prefill_tokens":7115,"num_requests":1,"active_p_offloads":0,"cached_blocks":382,"cache_hit":0,"new_prefill":11128,"score_linear":57803.0,"score_lmetric":18243},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":27927,"ongoing_decode_tokens":0,"pending_prefill_tokens":27927,"num_requests":1,"active_p_offloads":0,"cached_blocks":254,"cache_hit":0,"new_prefill":11128,"score_linear":27927.0,"score_lmetric":39055}],"chosen_score_linear":15283.0,"chosen_score_lmetric":11128,"t_first_token":281663.680324808,"t_first_token_unix":1779712528.6942687,"t_done":281663.897951118,"t_done_unix":1779712528.9118943},{"request_id":"1256273:4:1259632:60","session_id":"1256273","input_length":12270,"t_proxy_recv":281625.051032826,"t_decision_unix":1779712490.064974,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":7680,"affinity_cache_ratio":0.6259168704156479,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":4590,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":12270,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":27387,"pending_prefill_tokens":17650,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":12270,"score_linear":80365.0,"score_lmetric":59840},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":12270,"score_linear":68131.0,"score_lmetric":80401},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":9250,"ongoing_decode_tokens":9250,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":12270,"score_linear":9250.0,"score_lmetric":12270},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":12270,"score_linear":90488.0,"score_lmetric":102758},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":112793,"ongoing_decode_tokens":0,"pending_prefill_tokens":112793,"num_requests":1,"active_p_offloads":0,"cached_blocks":36,"cache_hit":7680,"new_prefill":4590,"score_linear":105113.0,"score_lmetric":117383},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":14291,"ongoing_decode_tokens":14291,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":12270,"score_linear":14291.0,"score_lmetric":12270},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":12270,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":105113.0,"chosen_score_lmetric":117383,"t_first_token":281664.048439814,"t_first_token_unix":1779712529.0623834,"t_done":281665.08906358,"t_done_unix":1779712530.1030066},{"request_id":"1258859:1:1258859:57","session_id":"1258859","input_length":112793,"t_proxy_recv":281622.462709728,"t_decision_unix":1779712487.476651,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":112793,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":112793,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":27387,"pending_prefill_tokens":17650,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":112793,"score_linear":80365.0,"score_lmetric":260886},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":112793,"score_linear":68131.0,"score_lmetric":180924},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":112793,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":112793,"score_linear":90488.0,"score_lmetric":203281},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":112793,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":112793,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":112793,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281663.823624514,"t_first_token_unix":1779712528.8375685,"t_done":281665.205469924,"t_done_unix":1779712530.2194135},{"request_id":"1263727:2:1270052:109","session_id":"1263727","input_length":57803,"t_proxy_recv":281661.202927823,"t_decision_unix":1779712526.2168694,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":50688,"affinity_cache_ratio":0.876909502966974,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":50688,"estimated_new_tokens":7115,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":57803,"score_linear":135235.0,"score_lmetric":342044},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":57803,"score_linear":102074.0,"score_lmetric":156293},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":57803,"score_linear":126924.0,"score_lmetric":165783},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":57803,"score_linear":15283.0,"score_lmetric":57803},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":57803,"score_linear":27639.0,"score_lmetric":57803},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":57803,"score_linear":125063.0,"score_lmetric":350372},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":382,"cache_hit":50688,"new_prefill":7115,"score_linear":-50688.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":32151,"ongoing_decode_tokens":32151,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":252,"cache_hit":0,"new_prefill":57803,"score_linear":32151.0,"score_lmetric":115606}],"chosen_score_linear":-50688.0,"chosen_score_lmetric":0,"t_first_token":281663.764280877,"t_first_token_unix":1779712528.7782247,"t_done":281666.883980931,"t_done_unix":1779712531.8979247},{"request_id":"1270340:1:1270340:111","session_id":"1270340","input_length":27927,"t_proxy_recv":281662.208952663,"t_decision_unix":1779712527.2228942,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":27927,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":27927,"score_linear":135235.0,"score_lmetric":282292},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":27927,"score_linear":102074.0,"score_lmetric":126417},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":27927,"score_linear":126924.0,"score_lmetric":135907},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":27927,"score_linear":15283.0,"score_lmetric":27927},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":27927,"score_linear":27639.0,"score_lmetric":27927},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":27927,"score_linear":125063.0,"score_lmetric":290620},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":57803,"ongoing_decode_tokens":0,"pending_prefill_tokens":7115,"num_requests":1,"active_p_offloads":0,"cached_blocks":382,"cache_hit":0,"new_prefill":27927,"score_linear":57803.0,"score_lmetric":35042},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":254,"cache_hit":0,"new_prefill":27927,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281666.531313641,"t_first_token_unix":1779712531.5452566,"t_done":281666.93822855,"t_done_unix":1779712531.9521718},{"request_id":"1271847:1:1271847:123","session_id":"1271847","input_length":2524,"t_proxy_recv":281667.465461881,"t_decision_unix":1779712532.4794033,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":5048,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":2524,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":2524,"score_linear":135235.0,"score_lmetric":231486},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":2524,"score_linear":102074.0,"score_lmetric":101014},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":2524,"score_linear":150574.0,"score_lmetric":240660},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":37753,"ongoing_decode_tokens":16034,"pending_prefill_tokens":21719,"num_requests":3,"active_p_offloads":0,"cached_blocks":379,"cache_hit":0,"new_prefill":2524,"score_linear":37753.0,"score_lmetric":72729},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":2524,"score_linear":121069.0,"score_lmetric":191908},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":130878,"ongoing_decode_tokens":113172,"pending_prefill_tokens":17706,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":2524,"score_linear":130878.0,"score_lmetric":40460},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":62237,"ongoing_decode_tokens":62237,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":2524,"score_linear":62237.0,"score_lmetric":7572},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":61097,"ongoing_decode_tokens":61097,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":308,"cache_hit":0,"new_prefill":2524,"score_linear":61097.0,"score_lmetric":5048}],"chosen_score_linear":61097.0,"chosen_score_lmetric":5048,"t_first_token":281667.664392627,"t_first_token_unix":1779712532.678336,"t_done":281667.935758922,"t_done_unix":1779712532.9497018},{"request_id":"1268831:2:1271087:116","session_id":"1268831","input_length":19448,"t_proxy_recv":281665.028954369,"t_decision_unix":1779712530.0428956,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":12288,"affinity_cache_ratio":0.6318387494858083,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":12288,"estimated_new_tokens":7160,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":19448,"score_linear":135235.0,"score_lmetric":265334},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":19448,"score_linear":102074.0,"score_lmetric":117938},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":19448,"score_linear":126924.0,"score_lmetric":127428},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":379,"cache_hit":0,"new_prefill":19448,"score_linear":15283.0,"score_lmetric":19448},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":19448,"score_linear":121069.0,"score_lmetric":225756},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":125063,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":19448,"score_linear":125063.0,"score_lmetric":38896},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":86188,"ongoing_decode_tokens":57803,"pending_prefill_tokens":10465,"num_requests":2,"active_p_offloads":0,"cached_blocks":382,"cache_hit":12288,"new_prefill":7160,"score_linear":73900.0,"score_lmetric":35250},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60701,"ongoing_decode_tokens":0,"pending_prefill_tokens":32029,"num_requests":2,"active_p_offloads":0,"cached_blocks":254,"cache_hit":0,"new_prefill":19448,"score_linear":60701.0,"score_lmetric":102954}],"chosen_score_linear":73900.0,"chosen_score_lmetric":35250,"t_first_token":281666.52856799,"t_first_token_unix":1779712531.5425115,"t_done":281668.724479822,"t_done_unix":1779712533.7384229},{"request_id":"1271991:1:1271991:124","session_id":"1271991","input_length":2851,"t_proxy_recv":281667.996194124,"t_decision_unix":1779712533.0101357,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":5702,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":2851,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":2851,"score_linear":135235.0,"score_lmetric":232140},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":2851,"score_linear":102074.0,"score_lmetric":101341},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":2851,"score_linear":150574.0,"score_lmetric":241314},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":37753,"ongoing_decode_tokens":16034,"pending_prefill_tokens":21719,"num_requests":3,"active_p_offloads":0,"cached_blocks":379,"cache_hit":0,"new_prefill":2851,"score_linear":37753.0,"score_lmetric":73710},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":2851,"score_linear":121069.0,"score_lmetric":192562},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":130878,"ongoing_decode_tokens":130878,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":2851,"score_linear":130878.0,"score_lmetric":5702},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":62237,"ongoing_decode_tokens":62237,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":2851,"score_linear":62237.0,"score_lmetric":8553},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":61097,"ongoing_decode_tokens":61097,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":312,"cache_hit":0,"new_prefill":2851,"score_linear":61097.0,"score_lmetric":5702}],"chosen_score_linear":130878.0,"chosen_score_lmetric":5702,"t_first_token":281668.401053353,"t_first_token_unix":1779712533.4149966,"t_done":281668.809928706,"t_done_unix":1779712533.8238719},{"request_id":"1271328:1:1271328:117","session_id":"1271328","input_length":17706,"t_proxy_recv":281665.752354533,"t_decision_unix":1779712530.7662952,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":17706,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":17706,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":17706,"score_linear":135235.0,"score_lmetric":261850},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":17706,"score_linear":102074.0,"score_lmetric":116196},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":17706,"score_linear":126924.0,"score_lmetric":125686},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":379,"cache_hit":0,"new_prefill":17706,"score_linear":15283.0,"score_lmetric":17706},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":17706,"score_linear":121069.0,"score_lmetric":222272},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":17706,"score_linear":113172.0,"score_lmetric":17706},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":105636,"ongoing_decode_tokens":86188,"pending_prefill_tokens":7160,"num_requests":3,"active_p_offloads":0,"cached_blocks":382,"cache_hit":0,"new_prefill":17706,"score_linear":105636.0,"score_lmetric":74598},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60701,"ongoing_decode_tokens":0,"pending_prefill_tokens":32029,"num_requests":2,"active_p_offloads":0,"cached_blocks":254,"cache_hit":0,"new_prefill":17706,"score_linear":60701.0,"score_lmetric":99470}],"chosen_score_linear":113172.0,"chosen_score_lmetric":17706,"t_first_token":281667.605819964,"t_first_token_unix":1779712532.6197631,"t_done":281669.033377037,"t_done_unix":1779712534.0473204},{"request_id":"1272085:1:1272085:125","session_id":"1272085","input_length":5302,"t_proxy_recv":281668.332872187,"t_decision_unix":1779712533.3468132,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.25,"fallback_score":10604,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":5302,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":5302,"score_linear":135235.0,"score_lmetric":237042},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":5302,"score_linear":102074.0,"score_lmetric":103792},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":5302,"score_linear":150574.0,"score_lmetric":246216},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":37753,"ongoing_decode_tokens":16034,"pending_prefill_tokens":21719,"num_requests":3,"active_p_offloads":0,"cached_blocks":379,"cache_hit":0,"new_prefill":5302,"score_linear":37753.0,"score_lmetric":81063},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":5302,"score_linear":121069.0,"score_lmetric":197464},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":133729,"ongoing_decode_tokens":130878,"pending_prefill_tokens":2851,"num_requests":3,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":5302,"score_linear":133729.0,"score_lmetric":24459},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":62237,"ongoing_decode_tokens":62237,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":5302,"score_linear":62237.0,"score_lmetric":15906},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":61097,"ongoing_decode_tokens":61097,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":312,"cache_hit":0,"new_prefill":5302,"score_linear":61097.0,"score_lmetric":10604}],"chosen_score_linear":61097.0,"chosen_score_lmetric":10604,"t_first_token":281668.686833817,"t_first_token_unix":1779712533.7007773,"t_done":281669.43501276,"t_done_unix":1779712534.448956},{"request_id":"1271459:1:1271459:118","session_id":"1271459","input_length":751,"t_proxy_recv":281666.169397829,"t_decision_unix":1779712531.183339,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":751,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":751,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":751,"score_linear":135235.0,"score_lmetric":227940},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":751,"score_linear":102074.0,"score_lmetric":99241},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":751,"score_linear":126924.0,"score_lmetric":108731},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":379,"cache_hit":0,"new_prefill":751,"score_linear":15283.0,"score_lmetric":751},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":751,"score_linear":121069.0,"score_lmetric":188362},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":130878,"ongoing_decode_tokens":113172,"pending_prefill_tokens":17706,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":751,"score_linear":130878.0,"score_lmetric":36914},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":105636,"ongoing_decode_tokens":86188,"pending_prefill_tokens":7160,"num_requests":3,"active_p_offloads":0,"cached_blocks":382,"cache_hit":0,"new_prefill":751,"score_linear":105636.0,"score_lmetric":23733},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60701,"ongoing_decode_tokens":0,"pending_prefill_tokens":32029,"num_requests":2,"active_p_offloads":0,"cached_blocks":254,"cache_hit":0,"new_prefill":751,"score_linear":60701.0,"score_lmetric":65560}],"chosen_score_linear":15283.0,"chosen_score_lmetric":751,"t_first_token":281666.26605763,"t_first_token_unix":1779712531.280001,"t_done":281669.755717316,"t_done_unix":1779712534.7696605},{"request_id":"1270340:2:1271641:121","session_id":"1270340","input_length":28323,"t_proxy_recv":281666.943411658,"t_decision_unix":1779712531.957353,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":27648,"affinity_cache_ratio":0.9761677788369876,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":27648,"estimated_new_tokens":675,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":28323,"score_linear":135235.0,"score_lmetric":283084},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":28323,"score_linear":102074.0,"score_lmetric":126813},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28323,"score_linear":150574.0,"score_lmetric":292258},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":37753,"ongoing_decode_tokens":16034,"pending_prefill_tokens":21719,"num_requests":3,"active_p_offloads":0,"cached_blocks":379,"cache_hit":0,"new_prefill":28323,"score_linear":37753.0,"score_lmetric":150126},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":28323,"score_linear":121069.0,"score_lmetric":243506},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":130878,"ongoing_decode_tokens":113172,"pending_prefill_tokens":17706,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":28323,"score_linear":130878.0,"score_lmetric":92058},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":62237,"ongoing_decode_tokens":62237,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":28323,"score_linear":62237.0,"score_lmetric":84969},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":32774,"ongoing_decode_tokens":32774,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":308,"cache_hit":27648,"new_prefill":675,"score_linear":5126.0,"score_lmetric":675}],"chosen_score_linear":5126.0,"chosen_score_lmetric":675,"t_first_token":281667.113769397,"t_first_token_unix":1779712532.1277125,"t_done":281669.920376963,"t_done_unix":1779712534.9343202},{"request_id":"1266668:2:1270661:115","session_id":"1266668","input_length":28385,"t_proxy_recv":281663.39215505,"t_decision_unix":1779712528.4060965,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":17920,"affinity_cache_ratio":0.6313193588162762,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":17920,"estimated_new_tokens":10465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":24289,"score_linear":131139.0,"score_lmetric":275016},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":28385,"score_linear":102074.0,"score_lmetric":126875},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28385,"score_linear":126924.0,"score_lmetric":136365},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":26411,"ongoing_decode_tokens":15283,"pending_prefill_tokens":11128,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28385,"score_linear":26411.0,"score_lmetric":79026},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":28385,"score_linear":121069.0,"score_lmetric":243630},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":28385,"score_linear":125063.0,"score_lmetric":291536},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":57803,"ongoing_decode_tokens":0,"pending_prefill_tokens":7115,"num_requests":1,"active_p_offloads":0,"cached_blocks":382,"cache_hit":17920,"new_prefill":10465,"score_linear":39883.0,"score_lmetric":17580},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60701,"ongoing_decode_tokens":0,"pending_prefill_tokens":32029,"num_requests":2,"active_p_offloads":0,"cached_blocks":254,"cache_hit":0,"new_prefill":28385,"score_linear":60701.0,"score_lmetric":120828}],"chosen_score_linear":39883.0,"chosen_score_lmetric":17580,"t_first_token":281665.575155602,"t_first_token_unix":1779712530.589099,"t_done":281670.416618792,"t_done_unix":1779712535.430562},{"request_id":"1271486:1:1271486:119","session_id":"1271486","input_length":21719,"t_proxy_recv":281666.268514862,"t_decision_unix":1779712531.2824564,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":43438,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":21719,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":21719,"score_linear":135235.0,"score_lmetric":269876},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":21719,"score_linear":102074.0,"score_lmetric":120209},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":21719,"score_linear":126924.0,"score_lmetric":129699},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":16034,"ongoing_decode_tokens":16034,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":379,"cache_hit":0,"new_prefill":21719,"score_linear":16034.0,"score_lmetric":43438},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":21719,"score_linear":121069.0,"score_lmetric":230298},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":130878,"ongoing_decode_tokens":113172,"pending_prefill_tokens":17706,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":21719,"score_linear":130878.0,"score_lmetric":78850},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":105636,"ongoing_decode_tokens":86188,"pending_prefill_tokens":7160,"num_requests":3,"active_p_offloads":0,"cached_blocks":382,"cache_hit":0,"new_prefill":21719,"score_linear":105636.0,"score_lmetric":86637},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60701,"ongoing_decode_tokens":0,"pending_prefill_tokens":32029,"num_requests":2,"active_p_offloads":0,"cached_blocks":254,"cache_hit":0,"new_prefill":21719,"score_linear":60701.0,"score_lmetric":107496}],"chosen_score_linear":16034.0,"chosen_score_lmetric":43438,"t_first_token":281668.621428285,"t_first_token_unix":1779712533.6353714,"t_done":281670.608344647,"t_done_unix":1779712535.6222892},{"request_id":"1269373:2:1271580:120","session_id":"1269373","input_length":14404,"t_proxy_recv":281666.58210536,"t_decision_unix":1779712531.596047,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":14336,"affinity_cache_ratio":0.995279089141905,"affinity_num_requests":3,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":68,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":10308,"score_linear":131139.0,"score_lmetric":247054},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":14404,"score_linear":102074.0,"score_lmetric":112894},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":14404,"score_linear":126924.0,"score_lmetric":122384},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":37753,"ongoing_decode_tokens":16034,"pending_prefill_tokens":21719,"num_requests":3,"active_p_offloads":0,"cached_blocks":379,"cache_hit":0,"new_prefill":14404,"score_linear":37753.0,"score_lmetric":108369},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":14404,"score_linear":121069.0,"score_lmetric":215668},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":130878,"ongoing_decode_tokens":113172,"pending_prefill_tokens":17706,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":14404,"score_linear":130878.0,"score_lmetric":64220},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":105636,"ongoing_decode_tokens":105636,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":382,"cache_hit":14336,"new_prefill":68,"score_linear":91300.0,"score_lmetric":204},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60701,"ongoing_decode_tokens":60701,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":254,"cache_hit":0,"new_prefill":14404,"score_linear":60701.0,"score_lmetric":28808}],"chosen_score_linear":91300.0,"chosen_score_lmetric":204,"t_first_token":281666.661456719,"t_first_token_unix":1779712531.6754,"t_done":281671.30347986,"t_done_unix":1779712536.317423},{"request_id":"1262354:6:1270643:114","session_id":"1262354","input_length":32774,"t_proxy_recv":281663.3153214,"t_decision_unix":1779712528.3292627,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":28672,"affinity_cache_ratio":0.8748398120461341,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":28672,"estimated_new_tokens":4102,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":32774,"score_linear":135235.0,"score_lmetric":291986},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":32774,"score_linear":102074.0,"score_lmetric":131264},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":32774,"score_linear":126924.0,"score_lmetric":140754},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":26411,"ongoing_decode_tokens":15283,"pending_prefill_tokens":11128,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":32774,"score_linear":26411.0,"score_lmetric":87804},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":32774,"score_linear":121069.0,"score_lmetric":252408},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":32774,"score_linear":125063.0,"score_lmetric":300314},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":57803,"ongoing_decode_tokens":0,"pending_prefill_tokens":7115,"num_requests":1,"active_p_offloads":0,"cached_blocks":382,"cache_hit":0,"new_prefill":32774,"score_linear":57803.0,"score_lmetric":39889},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":27927,"ongoing_decode_tokens":0,"pending_prefill_tokens":27927,"num_requests":1,"active_p_offloads":0,"cached_blocks":254,"cache_hit":28672,"new_prefill":4102,"score_linear":-745.0,"score_lmetric":32029}],"chosen_score_linear":-745.0,"chosen_score_lmetric":32029,"t_first_token":281666.534741029,"t_first_token_unix":1779712531.5486844,"t_done":281672.835710723,"t_done_unix":1779712537.849654},{"request_id":"1272313:1:1272313:126","session_id":"1272313","input_length":9718,"t_proxy_recv":281669.245109921,"t_decision_unix":1779712534.2590513,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":4076,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":2038,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":5622,"score_linear":131139.0,"score_lmetric":237682},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":9718,"score_linear":102074.0,"score_lmetric":108208},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":9718,"score_linear":150574.0,"score_lmetric":255048},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":37753,"ongoing_decode_tokens":37753,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":379,"cache_hit":0,"new_prefill":9718,"score_linear":37753.0,"score_lmetric":29154},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":9718,"score_linear":121069.0,"score_lmetric":206296},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":304,"cache_hit":0,"new_prefill":9718,"score_linear":113172.0,"score_lmetric":9718},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":42789,"ongoing_decode_tokens":42789,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":409,"cache_hit":7680,"new_prefill":2038,"score_linear":35109.0,"score_lmetric":4076},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":66399,"ongoing_decode_tokens":66399,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":312,"cache_hit":0,"new_prefill":9718,"score_linear":66399.0,"score_lmetric":29154}],"chosen_score_linear":35109.0,"chosen_score_lmetric":4076,"t_first_token":281669.487499805,"t_first_token_unix":1779712534.5014436,"t_done":281673.489253862,"t_done_unix":1779712538.5031972},{"request_id":"1272085:2:1273645:130","session_id":"1272085","input_length":8017,"t_proxy_recv":281673.964985771,"t_decision_unix":1779712538.9789271,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":5120,"affinity_cache_ratio":0.6386428838717725,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":5120,"estimated_new_tokens":2897,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":8017,"score_linear":135235.0,"score_lmetric":242472},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":8017,"score_linear":120167.0,"score_lmetric":228720},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":8017,"score_linear":150574.0,"score_lmetric":251646},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":37246,"ongoing_decode_tokens":37246,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":422,"cache_hit":0,"new_prefill":8017,"score_linear":37246.0,"score_lmetric":16034},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":8017,"score_linear":121069.0,"score_lmetric":202894},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":304,"cache_hit":0,"new_prefill":8017,"score_linear":113172.0,"score_lmetric":8017},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33711,"ongoing_decode_tokens":33711,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":435,"cache_hit":0,"new_prefill":8017,"score_linear":33711.0,"score_lmetric":8017},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":332,"cache_hit":5120,"new_prefill":2897,"score_linear":-5120.0,"score_lmetric":0}],"chosen_score_linear":-5120.0,"chosen_score_lmetric":0,"t_first_token":281674.254039216,"t_first_token_unix":1779712539.2679849,"t_done":281674.856572733,"t_done_unix":1779712539.8705158},{"request_id":"1260327:3:1273741:132","session_id":"1260327","input_length":31283,"t_proxy_recv":281674.367596597,"t_decision_unix":1779712539.3815377,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":30720,"affinity_cache_ratio":0.9820030048269028,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":30720,"estimated_new_tokens":563,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":31283,"score_linear":135235.0,"score_lmetric":289004},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":31283,"score_linear":120167.0,"score_lmetric":275252},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":31283,"score_linear":150574.0,"score_lmetric":298178},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":37246,"ongoing_decode_tokens":37246,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":422,"cache_hit":0,"new_prefill":31283,"score_linear":37246.0,"score_lmetric":62566},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":31283,"score_linear":121069.0,"score_lmetric":249426},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":138280,"ongoing_decode_tokens":113172,"pending_prefill_tokens":7700,"num_requests":2,"active_p_offloads":0,"cached_blocks":304,"cache_hit":0,"new_prefill":31283,"score_linear":138280.0,"score_lmetric":77966},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33711,"ongoing_decode_tokens":33711,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":435,"cache_hit":0,"new_prefill":31283,"score_linear":33711.0,"score_lmetric":31283},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":8017,"ongoing_decode_tokens":8017,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":332,"cache_hit":30720,"new_prefill":563,"score_linear":-22703.0,"score_lmetric":563}],"chosen_score_linear":-22703.0,"chosen_score_lmetric":563,"t_first_token":281674.560242209,"t_first_token_unix":1779712539.5741854,"t_done":281675.280634468,"t_done_unix":1779712540.2945776},{"request_id":"1266668:3:1273119:128","session_id":"1266668","input_length":33711,"t_proxy_recv":281672.020051932,"t_decision_unix":1779712537.0339932,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":28672,"affinity_cache_ratio":0.850523567974845,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":28672,"estimated_new_tokens":5039,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":29615,"score_linear":131139.0,"score_lmetric":285668},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":33711,"score_linear":120167.0,"score_lmetric":280108},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":33711,"score_linear":150574.0,"score_lmetric":303034},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":422,"cache_hit":0,"new_prefill":33711,"score_linear":15283.0,"score_lmetric":33711},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":33711,"score_linear":121069.0,"score_lmetric":254282},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":304,"cache_hit":0,"new_prefill":33711,"score_linear":113172.0,"score_lmetric":33711},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":9718,"ongoing_decode_tokens":9718,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":431,"cache_hit":28672,"new_prefill":5039,"score_linear":-18954.0,"score_lmetric":5039},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":32774,"ongoing_decode_tokens":32774,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":323,"cache_hit":0,"new_prefill":33711,"score_linear":32774.0,"score_lmetric":33711}],"chosen_score_linear":-18954.0,"chosen_score_lmetric":5039,"t_first_token":281673.161028807,"t_first_token_unix":1779712538.174972,"t_done":281675.797987492,"t_done_unix":1779712540.811931},{"request_id":"1258859:2:1264639:81","session_id":"1258859","input_length":113172,"t_proxy_recv":281665.223268074,"t_decision_unix":1779712530.2372096,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":112640,"affinity_cache_ratio":0.995299190612519,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":112640,"estimated_new_tokens":532,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":113172,"score_linear":135235.0,"score_lmetric":452782},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":113172,"score_linear":102074.0,"score_lmetric":211662},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":113172,"score_linear":126924.0,"score_lmetric":221152},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":379,"cache_hit":0,"new_prefill":113172,"score_linear":15283.0,"score_lmetric":113172},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":113172,"score_linear":121069.0,"score_lmetric":413204},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":265,"cache_hit":112640,"new_prefill":532,"score_linear":-112640.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":105636,"ongoing_decode_tokens":57803,"pending_prefill_tokens":17625,"num_requests":3,"active_p_offloads":0,"cached_blocks":382,"cache_hit":0,"new_prefill":113172,"score_linear":105636.0,"score_lmetric":392391},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60701,"ongoing_decode_tokens":0,"pending_prefill_tokens":32029,"num_requests":2,"active_p_offloads":0,"cached_blocks":254,"cache_hit":0,"new_prefill":113172,"score_linear":60701.0,"score_lmetric":290402}],"chosen_score_linear":-112640.0,"chosen_score_lmetric":0,"t_first_token":281665.597793492,"t_first_token_unix":1779712530.611737,"t_done":281676.84781009,"t_done_unix":1779712541.8617537},{"request_id":"1262727:1:1262727:72","session_id":"1262727","input_length":15283,"t_proxy_recv":281635.823767533,"t_decision_unix":1779712500.837709,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":15283,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":15283,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":0,"pending_prefill_tokens":1205,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":15283,"score_linear":55989.0,"score_lmetric":16488},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":15283,"score_linear":68131.0,"score_lmetric":83414},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":15283,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":15283,"score_linear":113053.0,"score_lmetric":215712},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":15283,"score_linear":125063.0,"score_lmetric":265332},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":15283,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21510,"ongoing_decode_tokens":0,"pending_prefill_tokens":21510,"num_requests":1,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":15283,"score_linear":21510.0,"score_lmetric":36793}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281637.234063684,"t_first_token_unix":1779712502.2480083,"t_done":281677.61261928,"t_done_unix":1779712542.6265626},{"request_id":"1274405:1:1274405:134","session_id":"1274405","input_length":6208,"t_proxy_recv":281676.767418307,"t_decision_unix":1779712541.7813597,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":6208,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":6208,"score_linear":135235.0,"score_lmetric":238854},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":6208,"score_linear":120167.0,"score_lmetric":225102},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":6208,"score_linear":150574.0,"score_lmetric":248028},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":37246,"ongoing_decode_tokens":37246,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":422,"cache_hit":0,"new_prefill":6208,"score_linear":37246.0,"score_lmetric":12416},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":6208,"score_linear":121069.0,"score_lmetric":199276},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":138280,"ongoing_decode_tokens":138280,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":304,"cache_hit":0,"new_prefill":6208,"score_linear":138280.0,"score_lmetric":12416},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":445,"cache_hit":0,"new_prefill":6208,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":338,"cache_hit":0,"new_prefill":6208,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281677.173657967,"t_first_token_unix":1779712542.1876016,"t_done":281677.651842957,"t_done_unix":1779712542.6657863},{"request_id":"1273203:1:1273203:129","session_id":"1273203","input_length":21963,"t_proxy_recv":281672.323208833,"t_decision_unix":1779712537.33715,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":9163,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":9163,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":21963,"score_linear":135235.0,"score_lmetric":270364},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":21963,"score_linear":120167.0,"score_lmetric":256612},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":21963,"score_linear":150574.0,"score_lmetric":279538},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":422,"cache_hit":12800,"new_prefill":9163,"score_linear":2483.0,"score_lmetric":9163},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":21963,"score_linear":121069.0,"score_lmetric":230786},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":304,"cache_hit":0,"new_prefill":21963,"score_linear":113172.0,"score_lmetric":21963},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":43429,"ongoing_decode_tokens":9718,"pending_prefill_tokens":5039,"num_requests":2,"active_p_offloads":0,"cached_blocks":431,"cache_hit":0,"new_prefill":21963,"score_linear":43429.0,"score_lmetric":54004},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":32774,"ongoing_decode_tokens":32774,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":323,"cache_hit":0,"new_prefill":21963,"score_linear":32774.0,"score_lmetric":21963}],"chosen_score_linear":2483.0,"chosen_score_lmetric":9163,"t_first_token":281673.704974213,"t_first_token_unix":1779712538.7189176,"t_done":281678.06581851,"t_done_unix":1779712543.0797617},{"request_id":"1271328:2:1273711:131","session_id":"1271328","input_length":25108,"t_proxy_recv":281674.228554316,"t_decision_unix":1779712539.242493,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":17408,"affinity_cache_ratio":0.6933248367054325,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":17408,"estimated_new_tokens":7700,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":25108,"score_linear":135235.0,"score_lmetric":276654},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":25108,"score_linear":120167.0,"score_lmetric":262902},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":25108,"score_linear":150574.0,"score_lmetric":285828},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":37246,"ongoing_decode_tokens":37246,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":422,"cache_hit":0,"new_prefill":25108,"score_linear":37246.0,"score_lmetric":50216},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":25108,"score_linear":121069.0,"score_lmetric":237076},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":304,"cache_hit":17408,"new_prefill":7700,"score_linear":95764.0,"score_lmetric":7700},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33711,"ongoing_decode_tokens":33711,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":435,"cache_hit":0,"new_prefill":25108,"score_linear":33711.0,"score_lmetric":25108},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":8017,"ongoing_decode_tokens":0,"pending_prefill_tokens":2897,"num_requests":1,"active_p_offloads":0,"cached_blocks":332,"cache_hit":0,"new_prefill":25108,"score_linear":8017.0,"score_lmetric":28005}],"chosen_score_linear":95764.0,"chosen_score_lmetric":7700,"t_first_token":281675.503417677,"t_first_token_unix":1779712540.5173612,"t_done":281679.027985377,"t_done_unix":1779712544.0419288},{"request_id":"1274890:1:1274890:135","session_id":"1274890","input_length":10506,"t_proxy_recv":281678.288882962,"t_decision_unix":1779712543.3028243,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":10506,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":10506,"score_linear":135235.0,"score_lmetric":247450},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":10506,"score_linear":120167.0,"score_lmetric":233698},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":10506,"score_linear":150574.0,"score_lmetric":256624},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":10506,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":10506,"score_linear":121069.0,"score_lmetric":207872},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":25108,"ongoing_decode_tokens":25108,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":10506,"score_linear":25108.0,"score_lmetric":10506},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":445,"cache_hit":0,"new_prefill":10506,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":350,"cache_hit":0,"new_prefill":10506,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281679.105365832,"t_first_token_unix":1779712544.1193094,"t_done":281679.358467751,"t_done_unix":1779712544.3724108},{"request_id":"1275433:1:1275433:137","session_id":"1275433","input_length":7794,"t_proxy_recv":281680.040973295,"t_decision_unix":1779712545.054914,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7794,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":7794,"score_linear":135235.0,"score_lmetric":242026},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":7794,"score_linear":120167.0,"score_lmetric":228274},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":7794,"score_linear":150574.0,"score_lmetric":251200},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":7794,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":7794,"score_linear":121069.0,"score_lmetric":202448},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":7794,"score_linear":114612.0,"score_lmetric":122406},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":445,"cache_hit":0,"new_prefill":7794,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":370,"cache_hit":0,"new_prefill":7794,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281680.582384047,"t_first_token_unix":1779712545.5963278,"t_done":281680.633428627,"t_done_unix":1779712545.6473715},{"request_id":"1275433:2:1276027:139","session_id":"1275433","input_length":7463,"t_proxy_recv":281682.316556002,"t_decision_unix":1779712547.3304973,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":7168,"affinity_cache_ratio":0.960471660190272,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":7168,"estimated_new_tokens":295,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":7463,"score_linear":135235.0,"score_lmetric":241364},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":7463,"score_linear":120167.0,"score_lmetric":227612},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":7463,"score_linear":150574.0,"score_lmetric":250538},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":7463,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":7463,"score_linear":121069.0,"score_lmetric":201786},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":7463,"score_linear":114612.0,"score_lmetric":122075},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":10639,"ongoing_decode_tokens":10639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":460,"cache_hit":7168,"new_prefill":295,"score_linear":3471.0,"score_lmetric":295},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":370,"cache_hit":0,"new_prefill":7463,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":3471.0,"chosen_score_lmetric":295,"t_first_token":281682.351129402,"t_first_token_unix":1779712547.3650727,"t_done":281682.364992526,"t_done_unix":1779712547.3789356},{"request_id":"1272313:2:1275926:138","session_id":"1272313","input_length":10639,"t_proxy_recv":281681.978359497,"t_decision_unix":1779712546.9923007,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":9728,"affinity_cache_ratio":0.914371651471003,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":9728,"estimated_new_tokens":911,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":6543,"score_linear":131139.0,"score_lmetric":239524},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":10639,"score_linear":120167.0,"score_lmetric":233964},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":10639,"score_linear":150574.0,"score_lmetric":256890},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":10639,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":10639,"score_linear":121069.0,"score_lmetric":208138},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":10639,"score_linear":114612.0,"score_lmetric":125251},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":460,"cache_hit":9728,"new_prefill":911,"score_linear":-9728.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":370,"cache_hit":0,"new_prefill":10639,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9728.0,"chosen_score_lmetric":0,"t_first_token":281682.133272061,"t_first_token_unix":1779712547.1472156,"t_done":281683.42733569,"t_done_unix":1779712548.4412792},{"request_id":"1264679:1:1264679:82","session_id":"1264679","input_length":110048,"t_proxy_recv":281642.931900334,"t_decision_unix":1779712507.9458413,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":110048,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":110048,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":110048,"score_linear":55989.0,"score_lmetric":110048},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":110048,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":110048,"score_linear":71575.0,"score_lmetric":332680},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":110048,"score_linear":113053.0,"score_lmetric":405242},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":110048,"score_linear":125063.0,"score_lmetric":454862},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":110048,"score_linear":50783.0,"score_lmetric":160831},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":110048,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281682.546797322,"t_first_token_unix":1779712547.5607412,"t_done":281683.650032701,"t_done_unix":1779712548.6639762},{"request_id":"1276111:1:1276111:140","session_id":"1276111","input_length":15783,"t_proxy_recv":281682.660141965,"t_decision_unix":1779712547.6740832,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":15783,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":135235,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":15783,"score_linear":135235.0,"score_lmetric":31566},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":15783,"score_linear":120167.0,"score_lmetric":244252},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":15783,"score_linear":150574.0,"score_lmetric":267178},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":15783,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":15783,"score_linear":121069.0,"score_lmetric":218426},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":15783,"score_linear":114612.0,"score_lmetric":130395},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":10639,"ongoing_decode_tokens":10639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":460,"cache_hit":0,"new_prefill":15783,"score_linear":10639.0,"score_lmetric":15783},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":370,"cache_hit":0,"new_prefill":15783,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281684.093831332,"t_first_token_unix":1779712549.107775,"t_done":281684.279696549,"t_done_unix":1779712549.2936397},{"request_id":"1276136:1:1276136:141","session_id":"1276136","input_length":17280,"t_proxy_recv":281682.744111581,"t_decision_unix":1779712547.7580528,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":17280,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":135235,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":17280,"score_linear":135235.0,"score_lmetric":34560},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":17280,"score_linear":120167.0,"score_lmetric":247246},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":17280,"score_linear":150574.0,"score_lmetric":270172},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":17280,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":17280,"score_linear":121069.0,"score_lmetric":221420},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":17280,"score_linear":114612.0,"score_lmetric":131892},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":10639,"ongoing_decode_tokens":10639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":460,"cache_hit":0,"new_prefill":17280,"score_linear":10639.0,"score_lmetric":17280},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":15783,"ongoing_decode_tokens":0,"pending_prefill_tokens":15783,"num_requests":1,"active_p_offloads":0,"cached_blocks":370,"cache_hit":0,"new_prefill":17280,"score_linear":15783.0,"score_lmetric":33063}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281684.397785735,"t_first_token_unix":1779712549.411729,"t_done":281684.778167286,"t_done_unix":1779712549.7921107},{"request_id":"1274405:2:1276717:144","session_id":"1274405","input_length":6360,"t_proxy_recv":281684.774083767,"t_decision_unix":1779712549.7880251,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":6144,"affinity_cache_ratio":0.9660377358490566,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":6144,"estimated_new_tokens":216,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":6360,"score_linear":25187.0,"score_lmetric":6360},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":6360,"score_linear":120167.0,"score_lmetric":225406},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":6360,"score_linear":150574.0,"score_lmetric":248332},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":17280,"ongoing_decode_tokens":17280,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":6360,"score_linear":17280.0,"score_lmetric":6360},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":6360,"score_linear":121069.0,"score_lmetric":199580},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":6360,"score_linear":114612.0,"score_lmetric":120972},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58084,"ongoing_decode_tokens":8704,"pending_prefill_tokens":15588,"num_requests":2,"active_p_offloads":0,"cached_blocks":462,"cache_hit":0,"new_prefill":6360,"score_linear":58084.0,"score_lmetric":43896},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":400,"cache_hit":6144,"new_prefill":216,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-6144.0,"chosen_score_lmetric":0,"t_first_token":281684.811599188,"t_first_token_unix":1779712549.8255424,"t_done":281684.980560291,"t_done_unix":1779712549.9945035},{"request_id":"1271991:2:1276999:145","session_id":"1271991","input_length":11292,"t_proxy_recv":281685.761387318,"t_decision_unix":1779712550.7753289,"policy":"unified","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":3,"affinity_cache_hit":2560,"affinity_cache_ratio":0.2267091746369111,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":11292,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":11292,"score_linear":25187.0,"score_lmetric":11292},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":11292,"score_linear":120167.0,"score_lmetric":235270},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":11292,"score_linear":150574.0,"score_lmetric":258196},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":508,"cache_hit":0,"new_prefill":11292,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":11292,"score_linear":121069.0,"score_lmetric":209444},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":2560,"new_prefill":8732,"score_linear":112052.0,"score_lmetric":123344},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58084,"ongoing_decode_tokens":8704,"pending_prefill_tokens":15588,"num_requests":2,"active_p_offloads":0,"cached_blocks":462,"cache_hit":0,"new_prefill":11292,"score_linear":58084.0,"score_lmetric":53760},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":400,"cache_hit":0,"new_prefill":11292,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281686.652720823,"t_first_token_unix":1779712551.6666641,"t_done":281686.814279543,"t_done_unix":1779712551.8282232},{"request_id":"1277533:1:1277533:150","session_id":"1277533","input_length":4729,"t_proxy_recv":281687.791960421,"t_decision_unix":1779712552.8059018,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":4729,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4729,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":4729,"score_linear":25187.0,"score_lmetric":4729},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":4729,"score_linear":120167.0,"score_lmetric":222144},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":4729,"score_linear":150574.0,"score_lmetric":245070},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":27221,"ongoing_decode_tokens":0,"pending_prefill_tokens":9813,"num_requests":1,"active_p_offloads":0,"cached_blocks":530,"cache_hit":0,"new_prefill":4729,"score_linear":27221.0,"score_lmetric":14542},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":4729,"score_linear":121069.0,"score_lmetric":196318},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":4729,"score_linear":114612.0,"score_lmetric":119341},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106829,"ongoing_decode_tokens":8704,"pending_prefill_tokens":30029,"num_requests":4,"active_p_offloads":0,"cached_blocks":462,"cache_hit":0,"new_prefill":4729,"score_linear":106829.0,"score_lmetric":139032},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":33524,"ongoing_decode_tokens":33524,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":400,"cache_hit":0,"new_prefill":4729,"score_linear":33524.0,"score_lmetric":4729}],"chosen_score_linear":33524.0,"chosen_score_lmetric":4729,"t_first_token":281688.105764818,"t_first_token_unix":1779712553.119708,"t_done":281688.320324387,"t_done_unix":1779712553.3342676},{"request_id":"1262354:7:1277104:146","session_id":"1262354","input_length":33524,"t_proxy_recv":281686.168963994,"t_decision_unix":1779712551.1829054,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":33280,"affinity_cache_ratio":0.992721632263453,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":33280,"estimated_new_tokens":244,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":33524,"score_linear":25187.0,"score_lmetric":33524},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":33524,"score_linear":120167.0,"score_lmetric":279734},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":33524,"score_linear":150574.0,"score_lmetric":302660},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":11292,"ongoing_decode_tokens":0,"pending_prefill_tokens":11292,"num_requests":1,"active_p_offloads":0,"cached_blocks":508,"cache_hit":0,"new_prefill":33524,"score_linear":11292.0,"score_lmetric":44816},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":33524,"score_linear":121069.0,"score_lmetric":253908},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":33524,"score_linear":114612.0,"score_lmetric":148136},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58084,"ongoing_decode_tokens":8704,"pending_prefill_tokens":15588,"num_requests":2,"active_p_offloads":0,"cached_blocks":462,"cache_hit":0,"new_prefill":33524,"score_linear":58084.0,"score_lmetric":98224},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":400,"cache_hit":33280,"new_prefill":244,"score_linear":-33280.0,"score_lmetric":0}],"chosen_score_linear":-33280.0,"chosen_score_lmetric":0,"t_first_token":281686.241547359,"t_first_token_unix":1779712551.2554908,"t_done":281689.070445321,"t_done_unix":1779712554.0843885},{"request_id":"1277428:1:1277428:149","session_id":"1277428","input_length":27221,"t_proxy_recv":281687.363185259,"t_decision_unix":1779712552.377127,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":17408,"estimated_new_tokens":9813,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":27221,"score_linear":25187.0,"score_lmetric":27221},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":3584,"new_prefill":23637,"score_linear":116583.0,"score_lmetric":259960},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":27221,"score_linear":150574.0,"score_lmetric":290054},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":530,"cache_hit":17408,"new_prefill":9813,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":27221,"score_linear":121069.0,"score_lmetric":241302},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":27221,"score_linear":114612.0,"score_lmetric":141833},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106829,"ongoing_decode_tokens":8704,"pending_prefill_tokens":30029,"num_requests":4,"active_p_offloads":0,"cached_blocks":462,"cache_hit":0,"new_prefill":27221,"score_linear":106829.0,"score_lmetric":229000},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":33524,"ongoing_decode_tokens":33524,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":400,"cache_hit":0,"new_prefill":27221,"score_linear":33524.0,"score_lmetric":27221}],"chosen_score_linear":-17408.0,"chosen_score_lmetric":0,"t_first_token":281689.140388915,"t_first_token_unix":1779712554.1543324,"t_done":281689.975930802,"t_done_unix":1779712554.989874},{"request_id":"1273203:2:1277744:152","session_id":"1273203","input_length":22573,"t_proxy_recv":281688.516695875,"t_decision_unix":1779712553.530637,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":22016,"affinity_cache_ratio":0.9753245027244939,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":22016,"estimated_new_tokens":557,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34071,"ongoing_decode_tokens":34071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":22573,"score_linear":34071.0,"score_lmetric":45146},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":22573,"score_linear":120167.0,"score_lmetric":257832},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":22573,"score_linear":150574.0,"score_lmetric":280758},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":27221,"ongoing_decode_tokens":0,"pending_prefill_tokens":9813,"num_requests":1,"active_p_offloads":0,"cached_blocks":530,"cache_hit":22016,"new_prefill":557,"score_linear":5205.0,"score_lmetric":10370},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":22573,"score_linear":121069.0,"score_lmetric":232006},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":22573,"score_linear":114612.0,"score_lmetric":137185},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106829,"ongoing_decode_tokens":8704,"pending_prefill_tokens":30029,"num_requests":4,"active_p_offloads":0,"cached_blocks":462,"cache_hit":0,"new_prefill":22573,"score_linear":106829.0,"score_lmetric":210408},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":33524,"ongoing_decode_tokens":33524,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":409,"cache_hit":0,"new_prefill":22573,"score_linear":33524.0,"score_lmetric":22573}],"chosen_score_linear":5205.0,"chosen_score_lmetric":10370,"t_first_token":281689.177765088,"t_first_token_unix":1779712554.1917083,"t_done":281691.092484501,"t_done_unix":1779712556.1064277},{"request_id":"1271459:2:1277544:151","session_id":"1271459","input_length":8884,"t_proxy_recv":281687.82991291,"t_decision_unix":1779712552.8438544,"policy":"unified","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":0,"affinity_cache_hit":512,"affinity_cache_ratio":0.05763169743358847,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":8884,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":8884,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":8884,"score_linear":25187.0,"score_lmetric":8884},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":8884,"score_linear":120167.0,"score_lmetric":230454},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":8884,"score_linear":150574.0,"score_lmetric":253380},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":27221,"ongoing_decode_tokens":0,"pending_prefill_tokens":9813,"num_requests":1,"active_p_offloads":0,"cached_blocks":530,"cache_hit":512,"new_prefill":8372,"score_linear":26709.0,"score_lmetric":18185},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":8884,"score_linear":121069.0,"score_lmetric":204628},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":8884,"score_linear":114612.0,"score_lmetric":123496},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106829,"ongoing_decode_tokens":8704,"pending_prefill_tokens":30029,"num_requests":4,"active_p_offloads":0,"cached_blocks":462,"cache_hit":0,"new_prefill":8884,"score_linear":106829.0,"score_lmetric":155652},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":38253,"ongoing_decode_tokens":33524,"pending_prefill_tokens":4729,"num_requests":2,"active_p_offloads":0,"cached_blocks":400,"cache_hit":0,"new_prefill":8884,"score_linear":38253.0,"score_lmetric":27226}],"chosen_score_linear":25187.0,"chosen_score_lmetric":8884,"t_first_token":281688.512865688,"t_first_token_unix":1779712553.526809,"t_done":281691.917862377,"t_done_unix":1779712556.9318054},{"request_id":"1276498:1:1276498:142","session_id":"1276498","input_length":8704,"t_proxy_recv":281683.991505546,"t_decision_unix":1779712549.005447,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":8704,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":8704,"score_linear":25187.0,"score_lmetric":8704},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":8704,"score_linear":120167.0,"score_lmetric":230094},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":8704,"score_linear":150574.0,"score_lmetric":253020},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":17280,"ongoing_decode_tokens":0,"pending_prefill_tokens":17280,"num_requests":1,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":8704,"score_linear":17280.0,"score_lmetric":25984},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":8704,"score_linear":121069.0,"score_lmetric":204268},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":8704,"score_linear":114612.0,"score_lmetric":123316},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":462,"cache_hit":0,"new_prefill":8704,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":15783,"ongoing_decode_tokens":0,"pending_prefill_tokens":15783,"num_requests":1,"active_p_offloads":0,"cached_blocks":370,"cache_hit":0,"new_prefill":8704,"score_linear":15783.0,"score_lmetric":24487}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281684.632780737,"t_first_token_unix":1779712549.646724,"t_done":281692.24875246,"t_done_unix":1779712557.2626958},{"request_id":"1265942:3:1272453:127","session_id":"1265942","input_length":18093,"t_proxy_recv":281669.797825066,"t_decision_unix":1779712534.8117666,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":10240,"affinity_cache_ratio":0.5659647377438789,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":10240,"estimated_new_tokens":7853,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":18093,"score_linear":135235.0,"score_lmetric":262624},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":10240,"new_prefill":7853,"score_linear":91834.0,"score_lmetric":106343},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":18093,"score_linear":150574.0,"score_lmetric":271798},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":37002,"ongoing_decode_tokens":37002,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":380,"cache_hit":0,"new_prefill":18093,"score_linear":37002.0,"score_lmetric":36186},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":18093,"score_linear":121069.0,"score_lmetric":223046},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":304,"cache_hit":0,"new_prefill":18093,"score_linear":113172.0,"score_lmetric":18093},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":52507,"ongoing_decode_tokens":52507,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":409,"cache_hit":0,"new_prefill":18093,"score_linear":52507.0,"score_lmetric":54279},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":61097,"ongoing_decode_tokens":61097,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":322,"cache_hit":0,"new_prefill":18093,"score_linear":61097.0,"score_lmetric":36186}],"chosen_score_linear":91834.0,"chosen_score_lmetric":106343,"t_first_token":281691.479321028,"t_first_token_unix":1779712556.4932642,"t_done":281692.559062889,"t_done_unix":1779712557.5730064},{"request_id":"1260327:4:1277974:154","session_id":"1260327","input_length":31430,"t_proxy_recv":281689.415711,"t_decision_unix":1779712554.4296525,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9937002863506205,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":31232,"estimated_new_tokens":198,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34071,"ongoing_decode_tokens":34071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":31430,"score_linear":34071.0,"score_lmetric":62860},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":31430,"score_linear":120167.0,"score_lmetric":275546},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":31430,"score_linear":150574.0,"score_lmetric":298472},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49794,"ongoing_decode_tokens":49794,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":530,"cache_hit":0,"new_prefill":31430,"score_linear":49794.0,"score_lmetric":62860},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":31430,"score_linear":121069.0,"score_lmetric":249720},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":31430,"score_linear":114612.0,"score_lmetric":146042},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106829,"ongoing_decode_tokens":58084,"pending_prefill_tokens":14441,"num_requests":4,"active_p_offloads":0,"cached_blocks":462,"cache_hit":0,"new_prefill":31430,"score_linear":106829.0,"score_lmetric":183484},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":24222,"ongoing_decode_tokens":0,"pending_prefill_tokens":24222,"num_requests":1,"active_p_offloads":0,"cached_blocks":410,"cache_hit":31232,"new_prefill":198,"score_linear":-7010.0,"score_lmetric":24420}],"chosen_score_linear":-7010.0,"chosen_score_lmetric":24420,"t_first_token":281692.042778324,"t_first_token_unix":1779712557.0567214,"t_done":281692.583373171,"t_done_unix":1779712557.597316},{"request_id":"1270606:1:1270606:113","session_id":"1270606","input_length":93430,"t_proxy_recv":281663.218239612,"t_decision_unix":1779712528.232181,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":93430,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":93430,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":93430,"score_linear":135235.0,"score_lmetric":413298},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":93430,"score_linear":102074.0,"score_lmetric":191920},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":93430,"score_linear":126924.0,"score_lmetric":201410},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":26411,"ongoing_decode_tokens":15283,"pending_prefill_tokens":11128,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":93430,"score_linear":26411.0,"score_lmetric":209116},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":93430,"score_linear":27639.0,"score_lmetric":93430},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":93430,"score_linear":125063.0,"score_lmetric":421626},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":57803,"ongoing_decode_tokens":0,"pending_prefill_tokens":7115,"num_requests":1,"active_p_offloads":0,"cached_blocks":382,"cache_hit":0,"new_prefill":93430,"score_linear":57803.0,"score_lmetric":100545},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":27927,"ongoing_decode_tokens":0,"pending_prefill_tokens":27927,"num_requests":1,"active_p_offloads":0,"cached_blocks":254,"cache_hit":0,"new_prefill":93430,"score_linear":27927.0,"score_lmetric":121357}],"chosen_score_linear":27639.0,"chosen_score_lmetric":93430,"t_first_token":281692.238504284,"t_first_token_unix":1779712557.252448,"t_done":281694.291663751,"t_done_unix":1779712559.305607},{"request_id":"1276498:2:1278996:157","session_id":"1276498","input_length":16650,"t_proxy_recv":281692.946347784,"t_decision_unix":1779712557.960289,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":8704,"affinity_cache_ratio":0.5227627627627628,"affinity_num_requests":3,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":8704,"estimated_new_tokens":7946,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":16650,"score_linear":25187.0,"score_lmetric":16650},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":182055,"ongoing_decode_tokens":160826,"pending_prefill_tokens":3309,"num_requests":3,"active_p_offloads":0,"cached_blocks":300,"cache_hit":0,"new_prefill":16650,"score_linear":182055.0,"score_lmetric":59877},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":16650,"score_linear":150574.0,"score_lmetric":268912},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":550,"cache_hit":0,"new_prefill":16650,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":121069,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":16650,"score_linear":121069.0,"score_lmetric":33300},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":16650,"score_linear":232709.0,"score_lmetric":271390},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98125,"ongoing_decode_tokens":98125,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":479,"cache_hit":8704,"new_prefill":7946,"score_linear":89421.0,"score_lmetric":23838},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":24222,"ongoing_decode_tokens":24222,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":410,"cache_hit":0,"new_prefill":16650,"score_linear":24222.0,"score_lmetric":16650}],"chosen_score_linear":89421.0,"chosen_score_lmetric":23838,"t_first_token":281693.913153279,"t_first_token_unix":1779712558.927097,"t_done":281694.325028816,"t_done_unix":1779712559.3389719},{"request_id":"1265942:4:1274182:133","session_id":"1265942","input_length":21229,"t_proxy_recv":281692.573446764,"t_decision_unix":1779712557.5873883,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":17920,"affinity_cache_ratio":0.8441283150407461,"affinity_num_requests":2,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":17920,"estimated_new_tokens":3309,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":21229,"score_linear":25187.0,"score_lmetric":21229},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":160826,"ongoing_decode_tokens":160826,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":300,"cache_hit":17920,"new_prefill":3309,"score_linear":142906.0,"score_lmetric":6618},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":21229,"score_linear":150574.0,"score_lmetric":278070},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":550,"cache_hit":0,"new_prefill":21229,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":121069,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":21229,"score_linear":121069.0,"score_lmetric":42458},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":21229,"score_linear":232709.0,"score_lmetric":280548},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98125,"ongoing_decode_tokens":98125,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":479,"cache_hit":0,"new_prefill":21229,"score_linear":98125.0,"score_lmetric":63687},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":55652,"ongoing_decode_tokens":55652,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":410,"cache_hit":0,"new_prefill":21229,"score_linear":55652.0,"score_lmetric":42458}],"chosen_score_linear":142906.0,"chosen_score_lmetric":6618,"t_first_token":281693.13719543,"t_first_token_unix":1779712558.1511388,"t_done":281694.395324076,"t_done_unix":1779712559.409267},{"request_id":"1268630:1:1268630:98","session_id":"1268630","input_length":102074,"t_proxy_recv":281656.554345638,"t_decision_unix":1779712521.5682867,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":3584,"estimated_new_tokens":98490,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":102074,"score_linear":135235.0,"score_lmetric":430586},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":285,"cache_hit":3584,"new_prefill":98490,"score_linear":-3584.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":102074,"score_linear":126924.0,"score_lmetric":210054},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":4608,"new_prefill":97466,"score_linear":10675.0,"score_lmetric":97466},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":102074,"score_linear":27639.0,"score_lmetric":102074},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":102074,"score_linear":125063.0,"score_lmetric":438914},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":341,"cache_hit":0,"new_prefill":102074,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":206,"cache_hit":0,"new_prefill":102074,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-3584.0,"chosen_score_lmetric":0,"t_first_token":281691.478223696,"t_first_token_unix":1779712556.492167,"t_done":281694.73195138,"t_done_unix":1779712559.7458951},{"request_id":"1268831:3:1277112:147","session_id":"1268831","input_length":31845,"t_proxy_recv":281686.208239591,"t_decision_unix":1779712551.222181,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":19456,"affinity_cache_ratio":0.6109593342753965,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":19456,"estimated_new_tokens":12389,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":31845,"score_linear":25187.0,"score_lmetric":31845},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":31845,"score_linear":120167.0,"score_lmetric":276376},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":31845,"score_linear":150574.0,"score_lmetric":299302},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":11292,"ongoing_decode_tokens":0,"pending_prefill_tokens":11292,"num_requests":1,"active_p_offloads":0,"cached_blocks":508,"cache_hit":0,"new_prefill":31845,"score_linear":11292.0,"score_lmetric":43137},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":31845,"score_linear":121069.0,"score_lmetric":250550},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":31845,"score_linear":114612.0,"score_lmetric":146457},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58084,"ongoing_decode_tokens":8704,"pending_prefill_tokens":15588,"num_requests":2,"active_p_offloads":0,"cached_blocks":462,"cache_hit":19456,"new_prefill":12389,"score_linear":38628.0,"score_lmetric":55954},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":33524,"ongoing_decode_tokens":0,"pending_prefill_tokens":244,"num_requests":1,"active_p_offloads":0,"cached_blocks":400,"cache_hit":0,"new_prefill":31845,"score_linear":33524.0,"score_lmetric":32089}],"chosen_score_linear":38628.0,"chosen_score_lmetric":55954,"t_first_token":281691.523205955,"t_first_token_unix":1779712556.5371494,"t_done":281695.459723509,"t_done_unix":1779712560.473667},{"request_id":"1272313:3:1279208:159","session_id":"1272313","input_length":11549,"t_proxy_recv":281693.60829358,"t_decision_unix":1779712558.6222348,"policy":"unified","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":3,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9309896960775825,"affinity_num_requests":4,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":11549,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":4096,"new_prefill":7453,"score_linear":21091.0,"score_lmetric":7453},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":182055,"ongoing_decode_tokens":182055,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":300,"cache_hit":0,"new_prefill":11549,"score_linear":182055.0,"score_lmetric":34647},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":11549,"score_linear":150574.0,"score_lmetric":258710},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":550,"cache_hit":0,"new_prefill":11549,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":121069,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":11549,"score_linear":121069.0,"score_lmetric":23098},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":11549,"score_linear":232709.0,"score_lmetric":261188},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":114775,"ongoing_decode_tokens":98125,"pending_prefill_tokens":7946,"num_requests":4,"active_p_offloads":0,"cached_blocks":479,"cache_hit":10752,"new_prefill":797,"score_linear":104023.0,"score_lmetric":34972},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":24222,"ongoing_decode_tokens":24222,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":410,"cache_hit":0,"new_prefill":11549,"score_linear":24222.0,"score_lmetric":11549}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281694.559430436,"t_first_token_unix":1779712559.573374,"t_done":281695.740281814,"t_done_unix":1779712560.7542248},{"request_id":"1279412:1:1279412:160","session_id":"1279412","input_length":20136,"t_proxy_recv":281694.412659519,"t_decision_unix":1779712559.426601,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":7336,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":7336,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":20136,"score_linear":25187.0,"score_lmetric":20136},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":160826,"ongoing_decode_tokens":160826,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":20136,"score_linear":160826.0,"score_lmetric":40272},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":20136,"score_linear":150574.0,"score_lmetric":275884},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":11549,"ongoing_decode_tokens":0,"pending_prefill_tokens":11549,"num_requests":1,"active_p_offloads":0,"cached_blocks":550,"cache_hit":0,"new_prefill":20136,"score_linear":11549.0,"score_lmetric":31685},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":465,"cache_hit":0,"new_prefill":20136,"score_linear":27639.0,"score_lmetric":20136},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":20136,"score_linear":232709.0,"score_lmetric":278362},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98125,"ongoing_decode_tokens":98125,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":494,"cache_hit":0,"new_prefill":20136,"score_linear":98125.0,"score_lmetric":60408},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":24222,"ongoing_decode_tokens":24222,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":410,"cache_hit":12800,"new_prefill":7336,"score_linear":11422.0,"score_lmetric":7336}],"chosen_score_linear":11422.0,"chosen_score_lmetric":7336,"t_first_token":281695.450233879,"t_first_token_unix":1779712560.4641771,"t_done":281697.4036977,"t_done_unix":1779712562.417641},{"request_id":"1268831:4:1279168:158","session_id":"1268831","input_length":32346,"t_proxy_recv":281695.466944996,"t_decision_unix":1779712560.4808865,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":31744,"affinity_cache_ratio":0.9813887343102702,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":31744,"estimated_new_tokens":602,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":32346,"score_linear":25187.0,"score_lmetric":32346},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":58752,"ongoing_decode_tokens":58752,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":32346,"score_linear":58752.0,"score_lmetric":32346},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":32346,"score_linear":150574.0,"score_lmetric":300304},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":11549,"ongoing_decode_tokens":11549,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":550,"cache_hit":0,"new_prefill":32346,"score_linear":11549.0,"score_lmetric":32346},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":465,"cache_hit":0,"new_prefill":32346,"score_linear":27639.0,"score_lmetric":32346},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":32346,"score_linear":232709.0,"score_lmetric":302782},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":66280,"ongoing_decode_tokens":66280,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":518,"cache_hit":31744,"new_prefill":602,"score_linear":34536.0,"score_lmetric":1204},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44358,"ongoing_decode_tokens":44358,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":410,"cache_hit":0,"new_prefill":32346,"score_linear":44358.0,"score_lmetric":64692}],"chosen_score_linear":34536.0,"chosen_score_lmetric":1204,"t_first_token":281695.629753187,"t_first_token_unix":1779712560.6436963,"t_done":281697.662765327,"t_done_unix":1779712562.6767087},{"request_id":"1253804:5:1278488:156","session_id":"1253804","input_length":58752,"t_proxy_recv":281691.096863743,"t_decision_unix":1779712556.110805,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":57344,"affinity_cache_ratio":0.9760348583877996,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57344,"estimated_new_tokens":1408,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34071,"ongoing_decode_tokens":34071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":58752,"score_linear":34071.0,"score_lmetric":117504},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":57344,"new_prefill":1408,"score_linear":62823.0,"score_lmetric":215502},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":58752,"score_linear":150574.0,"score_lmetric":353116},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":550,"cache_hit":3584,"new_prefill":55168,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":58752,"score_linear":121069.0,"score_lmetric":304364},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":58752,"score_linear":232709.0,"score_lmetric":355594},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106829,"ongoing_decode_tokens":58084,"pending_prefill_tokens":14441,"num_requests":4,"active_p_offloads":0,"cached_blocks":462,"cache_hit":0,"new_prefill":58752,"score_linear":106829.0,"score_lmetric":292772},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":55652,"ongoing_decode_tokens":0,"pending_prefill_tokens":24420,"num_requests":2,"active_p_offloads":0,"cached_blocks":410,"cache_hit":0,"new_prefill":58752,"score_linear":55652.0,"score_lmetric":166344}],"chosen_score_linear":62823.0,"chosen_score_lmetric":215502,"t_first_token":281691.93699437,"t_first_token_unix":1779712556.9509404,"t_done":281697.703324512,"t_done_unix":1779712562.7172678},{"request_id":"1277909:1:1277909:153","session_id":"1277909","input_length":24222,"t_proxy_recv":281689.220953464,"t_decision_unix":1779712554.2348948,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":24222,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34071,"ongoing_decode_tokens":34071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":371,"cache_hit":4096,"new_prefill":20126,"score_linear":29975.0,"score_lmetric":40252},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":24222,"score_linear":120167.0,"score_lmetric":261130},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":24222,"score_linear":150574.0,"score_lmetric":284056},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49794,"ongoing_decode_tokens":49794,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":530,"cache_hit":0,"new_prefill":24222,"score_linear":49794.0,"score_lmetric":48444},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":24222,"score_linear":121069.0,"score_lmetric":235304},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":24222,"score_linear":114612.0,"score_lmetric":138834},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106829,"ongoing_decode_tokens":58084,"pending_prefill_tokens":14441,"num_requests":4,"active_p_offloads":0,"cached_blocks":462,"cache_hit":7680,"new_prefill":16542,"score_linear":99149.0,"score_lmetric":123932},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":410,"cache_hit":0,"new_prefill":24222,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281692.040863149,"t_first_token_unix":1779712557.0548065,"t_done":281697.859438736,"t_done_unix":1779712562.8733819},{"request_id":"1262354:8:1280463:163","session_id":"1262354","input_length":33928,"t_proxy_recv":281697.726446144,"t_decision_unix":1779712562.7403877,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9959915114359821,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":33792,"estimated_new_tokens":136,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":33928,"score_linear":25187.0,"score_lmetric":33928},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":501,"cache_hit":0,"new_prefill":33928,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":33928,"score_linear":150574.0,"score_lmetric":303468},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":112836,"ongoing_decode_tokens":0,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":33928,"score_linear":112836.0,"score_lmetric":259736},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":465,"cache_hit":0,"new_prefill":33928,"score_linear":27639.0,"score_lmetric":33928},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":33928,"score_linear":232709.0,"score_lmetric":305946},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":66280,"ongoing_decode_tokens":66280,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":33928,"score_linear":66280.0,"score_lmetric":67856},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":24222,"ongoing_decode_tokens":24222,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":424,"cache_hit":33792,"new_prefill":136,"score_linear":-9570.0,"score_lmetric":136}],"chosen_score_linear":-9570.0,"chosen_score_lmetric":136,"t_first_token":281697.806223901,"t_first_token_unix":1779712562.820167,"t_done":281698.200013069,"t_done_unix":1779712563.2139564},{"request_id":"1266668:4:1276519:143","session_id":"1266668","input_length":49380,"t_proxy_recv":281684.050678268,"t_decision_unix":1779712549.0646193,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":33792,"affinity_cache_ratio":0.684325637910085,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":33792,"estimated_new_tokens":15588,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":4096,"new_prefill":45284,"score_linear":21091.0,"score_lmetric":45284},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":49380,"score_linear":120167.0,"score_lmetric":311446},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":49380,"score_linear":150574.0,"score_lmetric":334372},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":17280,"ongoing_decode_tokens":0,"pending_prefill_tokens":17280,"num_requests":1,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":49380,"score_linear":17280.0,"score_lmetric":66660},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":49380,"score_linear":121069.0,"score_lmetric":285620},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":49380,"score_linear":114612.0,"score_lmetric":163992},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":8704,"ongoing_decode_tokens":0,"pending_prefill_tokens":8704,"num_requests":1,"active_p_offloads":0,"cached_blocks":462,"cache_hit":33792,"new_prefill":15588,"score_linear":-25088.0,"score_lmetric":24292},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":15783,"ongoing_decode_tokens":0,"pending_prefill_tokens":15783,"num_requests":1,"active_p_offloads":0,"cached_blocks":370,"cache_hit":0,"new_prefill":49380,"score_linear":15783.0,"score_lmetric":65163}],"chosen_score_linear":-25088.0,"chosen_score_lmetric":24292,"t_first_token":281688.922155356,"t_first_token_unix":1779712553.9360988,"t_done":281698.702396855,"t_done_unix":1779712563.7163403},{"request_id":"1269373:3:1277365:148","session_id":"1269373","input_length":16900,"t_proxy_recv":281687.083805902,"t_decision_unix":1779712552.0977476,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":14848,"affinity_cache_ratio":0.8785798816568048,"affinity_num_requests":3,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":14848,"estimated_new_tokens":2052,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":4096,"new_prefill":12804,"score_linear":21091.0,"score_lmetric":12804},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":16900,"score_linear":120167.0,"score_lmetric":246486},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":16900,"score_linear":150574.0,"score_lmetric":269412},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":530,"cache_hit":0,"new_prefill":16900,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":16900,"score_linear":121069.0,"score_lmetric":220660},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":16900,"score_linear":114612.0,"score_lmetric":131512},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89929,"ongoing_decode_tokens":8704,"pending_prefill_tokens":27977,"num_requests":3,"active_p_offloads":0,"cached_blocks":462,"cache_hit":14848,"new_prefill":2052,"score_linear":75081.0,"score_lmetric":90087},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":33524,"ongoing_decode_tokens":33524,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":400,"cache_hit":0,"new_prefill":16900,"score_linear":33524.0,"score_lmetric":16900}],"chosen_score_linear":75081.0,"chosen_score_lmetric":90087,"t_first_token":281691.525805971,"t_first_token_unix":1779712556.5397491,"t_done":281699.155363496,"t_done_unix":1779712564.169307},{"request_id":"1268861:2:1280530:164","session_id":"1268861","input_length":12765,"t_proxy_recv":281697.942342976,"t_decision_unix":1779712562.956284,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":12288,"affinity_cache_ratio":0.9626321974148061,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":12288,"estimated_new_tokens":477,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":12765,"score_linear":25187.0,"score_lmetric":12765},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":501,"cache_hit":0,"new_prefill":12765,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":12765,"score_linear":150574.0,"score_lmetric":261142},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":112836,"ongoing_decode_tokens":0,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":12765,"score_linear":112836.0,"score_lmetric":217410},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":465,"cache_hit":12288,"new_prefill":477,"score_linear":15351.0,"score_lmetric":477},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":12765,"score_linear":232709.0,"score_lmetric":263620},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":66280,"ongoing_decode_tokens":66280,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":12765,"score_linear":66280.0,"score_lmetric":25530},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":33928,"ongoing_decode_tokens":33928,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":472,"cache_hit":0,"new_prefill":12765,"score_linear":33928.0,"score_lmetric":12765}],"chosen_score_linear":15351.0,"chosen_score_lmetric":477,"t_first_token":281698.002777775,"t_first_token_unix":1779712563.016721,"t_done":281701.267037371,"t_done_unix":1779712566.2809806},{"request_id":"1281887:1:1281887:171","session_id":"1281887","input_length":9370,"t_proxy_recv":281702.583238471,"t_decision_unix":1779712567.59718,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":9370,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":9370,"score_linear":25187.0,"score_lmetric":9370},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":501,"cache_hit":0,"new_prefill":9370,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":9370,"score_linear":150574.0,"score_lmetric":254352},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":9370,"score_linear":126737.0,"score_lmetric":323841},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":466,"cache_hit":0,"new_prefill":9370,"score_linear":27639.0,"score_lmetric":9370},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":9370,"score_linear":232709.0,"score_lmetric":256830},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":88294,"ongoing_decode_tokens":30262,"pending_prefill_tokens":8368,"num_requests":2,"active_p_offloads":0,"cached_blocks":555,"cache_hit":0,"new_prefill":9370,"score_linear":88294.0,"score_lmetric":35476},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60454,"ongoing_decode_tokens":60454,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":472,"cache_hit":0,"new_prefill":9370,"score_linear":60454.0,"score_lmetric":18740}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281703.298213513,"t_first_token_unix":1779712568.3121572,"t_done":281703.474556383,"t_done_unix":1779712568.4884996},{"request_id":"1282082:1:1282082:173","session_id":"1282082","input_length":4972,"t_proxy_recv":281703.333216926,"t_decision_unix":1779712568.3471582,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":4972,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4972,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":4972,"score_linear":25187.0,"score_lmetric":4972},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":9370,"ongoing_decode_tokens":9370,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":501,"cache_hit":0,"new_prefill":4972,"score_linear":9370.0,"score_lmetric":4972},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":4972,"score_linear":150574.0,"score_lmetric":245556},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":4972,"score_linear":126737.0,"score_lmetric":310647},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":466,"cache_hit":0,"new_prefill":4972,"score_linear":27639.0,"score_lmetric":4972},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":4972,"score_linear":232709.0,"score_lmetric":248034},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":88294,"ongoing_decode_tokens":30262,"pending_prefill_tokens":8368,"num_requests":2,"active_p_offloads":0,"cached_blocks":555,"cache_hit":0,"new_prefill":4972,"score_linear":88294.0,"score_lmetric":26680},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":102477,"ongoing_decode_tokens":60454,"pending_prefill_tokens":10791,"num_requests":3,"active_p_offloads":0,"cached_blocks":472,"cache_hit":0,"new_prefill":4972,"score_linear":102477.0,"score_lmetric":47289}],"chosen_score_linear":27639.0,"chosen_score_lmetric":4972,"t_first_token":281703.655746276,"t_first_token_unix":1779712568.66969,"t_done":281703.912372551,"t_done_unix":1779712568.926316},{"request_id":"1267915:1:1267915:96","session_id":"1267915","input_length":126924,"t_proxy_recv":281653.937400433,"t_decision_unix":1779712518.9513414,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":18944,"estimated_new_tokens":107980,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":126924,"score_linear":135235.0,"score_lmetric":480286},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":273,"cache_hit":0,"new_prefill":126924,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":358,"cache_hit":18944,"new_prefill":107980,"score_linear":-18944.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":126924,"score_linear":15283.0,"score_lmetric":126924},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":126924,"score_linear":27639.0,"score_lmetric":126924},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":126924,"score_linear":125063.0,"score_lmetric":488614},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":17466,"ongoing_decode_tokens":17466,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":126924,"score_linear":17466.0,"score_lmetric":126924},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28671,"ongoing_decode_tokens":0,"pending_prefill_tokens":6143,"num_requests":1,"active_p_offloads":0,"cached_blocks":194,"cache_hit":0,"new_prefill":126924,"score_linear":28671.0,"score_lmetric":133067}],"chosen_score_linear":-18944.0,"chosen_score_lmetric":0,"t_first_token":281704.182207819,"t_first_token_unix":1779712569.1961513,"t_done":281705.489057732,"t_done_unix":1779712570.503001},{"request_id":"1265128:2:1271655:122","session_id":"1265128","input_length":23650,"t_proxy_recv":281666.777622535,"t_decision_unix":1779712531.791564,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":13824,"affinity_cache_ratio":0.5845243128964059,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":13824,"estimated_new_tokens":9826,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":23650,"score_linear":135235.0,"score_lmetric":273738},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":23650,"score_linear":102074.0,"score_lmetric":122140},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":13824,"new_prefill":9826,"score_linear":113100.0,"score_lmetric":117806},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":37753,"ongoing_decode_tokens":16034,"pending_prefill_tokens":21719,"num_requests":3,"active_p_offloads":0,"cached_blocks":379,"cache_hit":0,"new_prefill":23650,"score_linear":37753.0,"score_lmetric":136107},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":23650,"score_linear":121069.0,"score_lmetric":234160},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":130878,"ongoing_decode_tokens":113172,"pending_prefill_tokens":17706,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":23650,"score_linear":130878.0,"score_lmetric":82712},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":120040,"ongoing_decode_tokens":120040,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":382,"cache_hit":0,"new_prefill":23650,"score_linear":120040.0,"score_lmetric":94600},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60701,"ongoing_decode_tokens":60701,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":254,"cache_hit":0,"new_prefill":23650,"score_linear":60701.0,"score_lmetric":47300}],"chosen_score_linear":113100.0,"chosen_score_lmetric":117806,"t_first_token":281704.674786421,"t_first_token_unix":1779712569.6887295,"t_done":281705.849249629,"t_done_unix":1779712570.8631926},{"request_id":"1260327:5:1281915:172","session_id":"1260327","input_length":42023,"t_proxy_recv":281702.702519431,"t_decision_unix":1779712567.7164607,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":31232,"affinity_cache_ratio":0.7432120505437498,"affinity_num_requests":2,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":31232,"estimated_new_tokens":10791,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":42023,"score_linear":25187.0,"score_lmetric":42023},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":9370,"ongoing_decode_tokens":0,"pending_prefill_tokens":9370,"num_requests":1,"active_p_offloads":0,"cached_blocks":501,"cache_hit":0,"new_prefill":42023,"score_linear":9370.0,"score_lmetric":51393},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":42023,"score_linear":150574.0,"score_lmetric":319658},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":42023,"score_linear":126737.0,"score_lmetric":421800},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":466,"cache_hit":0,"new_prefill":42023,"score_linear":27639.0,"score_lmetric":42023},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":42023,"score_linear":232709.0,"score_lmetric":322136},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":88294,"ongoing_decode_tokens":30262,"pending_prefill_tokens":8368,"num_requests":2,"active_p_offloads":0,"cached_blocks":555,"cache_hit":0,"new_prefill":42023,"score_linear":88294.0,"score_lmetric":100782},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60454,"ongoing_decode_tokens":60454,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":472,"cache_hit":31232,"new_prefill":10791,"score_linear":29222.0,"score_lmetric":21582}],"chosen_score_linear":29222.0,"chosen_score_lmetric":21582,"t_first_token":281705.460055088,"t_first_token_unix":1779712570.4739985,"t_done":281706.531427425,"t_done_unix":1779712571.5453703},{"request_id":"1262354:9:1281632:169","session_id":"1262354","input_length":34028,"t_proxy_recv":281701.72095194,"t_decision_unix":1779712566.7348928,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9930645350887505,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":33792,"estimated_new_tokens":236,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":34028,"score_linear":25187.0,"score_lmetric":34028},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":501,"cache_hit":0,"new_prefill":34028,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":34028,"score_linear":150574.0,"score_lmetric":303668},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":112836,"ongoing_decode_tokens":0,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":34028,"score_linear":112836.0,"score_lmetric":259936},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":466,"cache_hit":0,"new_prefill":34028,"score_linear":27639.0,"score_lmetric":34028},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":34028,"score_linear":232709.0,"score_lmetric":306146},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":88294,"ongoing_decode_tokens":30262,"pending_prefill_tokens":8368,"num_requests":2,"active_p_offloads":0,"cached_blocks":555,"cache_hit":0,"new_prefill":34028,"score_linear":88294.0,"score_lmetric":84792},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":26426,"ongoing_decode_tokens":26426,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":472,"cache_hit":33792,"new_prefill":236,"score_linear":-7366.0,"score_lmetric":236}],"chosen_score_linear":-7366.0,"chosen_score_lmetric":236,"t_first_token":281701.790033824,"t_first_token_unix":1779712566.803977,"t_done":281706.701152378,"t_done_unix":1779712571.7150955},{"request_id":"1269373:4:1280765:165","session_id":"1269373","input_length":30262,"t_proxy_recv":281699.166960349,"t_decision_unix":1779712564.180902,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":17408,"affinity_cache_ratio":0.5752428788579736,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":17408,"estimated_new_tokens":12854,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":4096,"new_prefill":26166,"score_linear":21091.0,"score_lmetric":26166},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":501,"cache_hit":0,"new_prefill":30262,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":30262,"score_linear":150574.0,"score_lmetric":296136},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":112836,"ongoing_decode_tokens":0,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":572,"cache_hit":7680,"new_prefill":22582,"score_linear":105156.0,"score_lmetric":237044},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":40404,"ongoing_decode_tokens":40404,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":465,"cache_hit":0,"new_prefill":30262,"score_linear":40404.0,"score_lmetric":60524},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":30262,"score_linear":232709.0,"score_lmetric":298614},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":555,"cache_hit":17408,"new_prefill":12854,"score_linear":-17408.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":472,"cache_hit":7680,"new_prefill":22582,"score_linear":-7680.0,"score_lmetric":0}],"chosen_score_linear":-17408.0,"chosen_score_lmetric":0,"t_first_token":281701.593712468,"t_first_token_unix":1779712566.607656,"t_done":281707.199185175,"t_done_unix":1779712572.213129},{"request_id":"1277533:2:1282872:175","session_id":"1277533","input_length":16299,"t_proxy_recv":281706.002747911,"t_decision_unix":1779712571.016689,"policy":"unified","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":2,"affinity_cache_hit":4608,"affinity_cache_ratio":0.2827167310877968,"affinity_num_requests":3,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":16299,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":16299,"score_linear":25187.0,"score_lmetric":16299},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":16299,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":566,"cache_hit":0,"new_prefill":16299,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":16299,"score_linear":126737.0,"score_lmetric":344628},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":16299,"score_linear":27639.0,"score_lmetric":16299},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":16299,"score_linear":232709.0,"score_lmetric":270688},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":88294,"ongoing_decode_tokens":88294,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":555,"cache_hit":0,"new_prefill":16299,"score_linear":88294.0,"score_lmetric":32598},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":102477,"ongoing_decode_tokens":102477,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":472,"cache_hit":4608,"new_prefill":11691,"score_linear":97869.0,"score_lmetric":35073}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281707.496032542,"t_first_token_unix":1779712572.509976,"t_done":281707.513326404,"t_done_unix":1779712572.5272694},{"request_id":"1266668:5:1281133:168","session_id":"1266668","input_length":58032,"t_proxy_recv":281700.267302813,"t_decision_unix":1779712565.2812436,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":49664,"affinity_cache_ratio":0.8558036945133719,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":8368,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":4096,"new_prefill":53936,"score_linear":21091.0,"score_lmetric":53936},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":501,"cache_hit":0,"new_prefill":58032,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":58032,"score_linear":150574.0,"score_lmetric":351676},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":112836,"ongoing_decode_tokens":0,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":572,"cache_hit":7680,"new_prefill":50352,"score_linear":105156.0,"score_lmetric":292584},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":40404,"ongoing_decode_tokens":40404,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":465,"cache_hit":0,"new_prefill":58032,"score_linear":40404.0,"score_lmetric":116064},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":58032,"score_linear":232709.0,"score_lmetric":354154},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":30262,"ongoing_decode_tokens":0,"pending_prefill_tokens":12854,"num_requests":1,"active_p_offloads":0,"cached_blocks":555,"cache_hit":49664,"new_prefill":8368,"score_linear":-19402.0,"score_lmetric":21222},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":26426,"ongoing_decode_tokens":0,"pending_prefill_tokens":6458,"num_requests":1,"active_p_offloads":0,"cached_blocks":472,"cache_hit":7680,"new_prefill":50352,"score_linear":18746.0,"score_lmetric":56810}],"chosen_score_linear":-19402.0,"chosen_score_lmetric":21222,"t_first_token":281704.373394798,"t_first_token_unix":1779712569.3873382,"t_done":281708.463569908,"t_done_unix":1779712573.4775136},{"request_id":"1242838:4:1265709:87","session_id":"1242838","input_length":25187,"t_proxy_recv":281646.408518373,"t_decision_unix":1779712511.4224594,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":22016,"affinity_cache_ratio":0.8741017191408266,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":22016,"estimated_new_tokens":3171,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":22016,"new_prefill":3171,"score_linear":88032.0,"score_lmetric":113219},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":25187,"score_linear":55989.0,"score_lmetric":25187},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":323,"cache_hit":0,"new_prefill":25187,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":25187,"score_linear":71575.0,"score_lmetric":162958},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":258,"cache_hit":0,"new_prefill":25187,"score_linear":27639.0,"score_lmetric":25187},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":25187,"score_linear":125063.0,"score_lmetric":285140},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":25187,"score_linear":50783.0,"score_lmetric":75970},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":0,"pending_prefill_tokens":23868,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":25187,"score_linear":45884.0,"score_lmetric":98110}],"chosen_score_linear":88032.0,"chosen_score_lmetric":113219,"t_first_token":281682.548315225,"t_first_token_unix":1779712547.5622582,"t_done":281708.825889704,"t_done_unix":1779712573.839833},{"request_id":"1283642:1:1283642:178","session_id":"1283642","input_length":8510,"t_proxy_recv":281708.438113726,"t_decision_unix":1779712573.4520547,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8510,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":8510,"score_linear":25187.0,"score_lmetric":8510},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":8510,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":8510,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":8510,"score_linear":126737.0,"score_lmetric":321261},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":8510,"score_linear":27639.0,"score_lmetric":8510},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":8510,"score_linear":232709.0,"score_lmetric":255110},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":88626,"ongoing_decode_tokens":88626,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":580,"cache_hit":0,"new_prefill":8510,"score_linear":88626.0,"score_lmetric":17020},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":26426,"ongoing_decode_tokens":26426,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":493,"cache_hit":0,"new_prefill":8510,"score_linear":26426.0,"score_lmetric":8510}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281709.06818726,"t_first_token_unix":1779712574.0821307,"t_done":281709.382162275,"t_done_unix":1779712574.3961055},{"request_id":"1279412:2:1281113:167","session_id":"1279412","input_length":26426,"t_proxy_recv":281700.210373979,"t_decision_unix":1779712565.2243152,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":19968,"affinity_cache_ratio":0.7556194656777416,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":6458,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":26426,"score_linear":25187.0,"score_lmetric":26426},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":501,"cache_hit":0,"new_prefill":26426,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":26426,"score_linear":150574.0,"score_lmetric":288464},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":112836,"ongoing_decode_tokens":0,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":26426,"score_linear":112836.0,"score_lmetric":244732},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":40404,"ongoing_decode_tokens":40404,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":465,"cache_hit":0,"new_prefill":26426,"score_linear":40404.0,"score_lmetric":52852},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":26426,"score_linear":232709.0,"score_lmetric":290942},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":30262,"ongoing_decode_tokens":0,"pending_prefill_tokens":12854,"num_requests":1,"active_p_offloads":0,"cached_blocks":555,"cache_hit":0,"new_prefill":26426,"score_linear":30262.0,"score_lmetric":39280},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":472,"cache_hit":19968,"new_prefill":6458,"score_linear":-19968.0,"score_lmetric":0}],"chosen_score_linear":-19968.0,"chosen_score_lmetric":0,"t_first_token":281701.335726062,"t_first_token_unix":1779712566.349672,"t_done":281710.262890017,"t_done_unix":1779712575.2768333},{"request_id":"1262354:10:1283773:180","session_id":"1262354","input_length":34282,"t_proxy_recv":281708.872512704,"t_decision_unix":1779712573.886454,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9857067849016977,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":33792,"estimated_new_tokens":490,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":399,"cache_hit":0,"new_prefill":34282,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":8510,"ongoing_decode_tokens":0,"pending_prefill_tokens":8510,"num_requests":1,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":34282,"score_linear":8510.0,"score_lmetric":42792},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":34282,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":34282,"score_linear":126737.0,"score_lmetric":398577},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":34282,"score_linear":27639.0,"score_lmetric":34282},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":34282,"score_linear":232709.0,"score_lmetric":306654},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":30594,"ongoing_decode_tokens":30594,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":34282,"score_linear":30594.0,"score_lmetric":34282},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":57391,"ongoing_decode_tokens":26426,"pending_prefill_tokens":6389,"num_requests":2,"active_p_offloads":0,"cached_blocks":493,"cache_hit":33792,"new_prefill":490,"score_linear":23599.0,"score_lmetric":13758}],"chosen_score_linear":23599.0,"chosen_score_lmetric":13758,"t_first_token":281709.857870319,"t_first_token_unix":1779712574.871815,"t_done":281710.511968388,"t_done_unix":1779712575.5259118},{"request_id":"1279412:3:1285475:187","session_id":"1279412","input_length":27077,"t_proxy_recv":281714.90830315,"t_decision_unix":1779712579.9222438,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":26624,"affinity_cache_ratio":0.9832699338922333,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":26624,"estimated_new_tokens":453,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":399,"cache_hit":0,"new_prefill":27077,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":535,"cache_hit":0,"new_prefill":27077,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":27077,"score_linear":49619.0,"score_lmetric":76696},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":27077,"score_linear":126737.0,"score_lmetric":376962},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":27077,"score_linear":27639.0,"score_lmetric":27077},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":27077,"score_linear":232709.0,"score_lmetric":292244},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":152913,"ongoing_decode_tokens":30594,"pending_prefill_tokens":31695,"num_requests":3,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":27077,"score_linear":152913.0,"score_lmetric":176316},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":65318,"ongoing_decode_tokens":65318,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":507,"cache_hit":26624,"new_prefill":453,"score_linear":38694.0,"score_lmetric":906}],"chosen_score_linear":38694.0,"chosen_score_lmetric":906,"t_first_token":281714.987726538,"t_first_token_unix":1779712580.00167,"t_done":281715.118952769,"t_done_unix":1779712580.1328957},{"request_id":"1277909:2:1283679:179","session_id":"1277909","input_length":30965,"t_proxy_recv":281708.545370367,"t_decision_unix":1779712573.5593119,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":24576,"affinity_cache_ratio":0.7936702728887454,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":24576,"estimated_new_tokens":6389,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":4096,"new_prefill":26869,"score_linear":21091.0,"score_lmetric":26869},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":8510,"ongoing_decode_tokens":0,"pending_prefill_tokens":8510,"num_requests":1,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":30965,"score_linear":8510.0,"score_lmetric":39475},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":30965,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":7680,"new_prefill":23285,"score_linear":119057.0,"score_lmetric":365586},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":30965,"score_linear":27639.0,"score_lmetric":30965},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":30965,"score_linear":232709.0,"score_lmetric":300020},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":30594,"ongoing_decode_tokens":30594,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":7680,"new_prefill":23285,"score_linear":22914.0,"score_lmetric":23285},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":26426,"ongoing_decode_tokens":26426,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":493,"cache_hit":24576,"new_prefill":6389,"score_linear":1850.0,"score_lmetric":6389}],"chosen_score_linear":1850.0,"chosen_score_lmetric":6389,"t_first_token":281709.821375911,"t_first_token_unix":1779712574.8353193,"t_done":281715.147691425,"t_done_unix":1779712580.1616344},{"request_id":"1285800:1:1285800:191","session_id":"1285800","input_length":4510,"t_proxy_recv":281716.016695845,"t_decision_unix":1779712581.0306368,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":4510,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4510,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":11,"ongoing_decode_tokens":11,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":399,"cache_hit":0,"new_prefill":4510,"score_linear":11.0,"score_lmetric":4510},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":14192,"ongoing_decode_tokens":0,"pending_prefill_tokens":6000,"num_requests":1,"active_p_offloads":0,"cached_blocks":535,"cache_hit":0,"new_prefill":4510,"score_linear":14192.0,"score_lmetric":10510},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":4510,"score_linear":49619.0,"score_lmetric":54129},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":4510,"score_linear":126737.0,"score_lmetric":309261},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":4510,"score_linear":27639.0,"score_lmetric":4510},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":4510,"score_linear":232709.0,"score_lmetric":247110},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":152913,"ongoing_decode_tokens":30594,"pending_prefill_tokens":31695,"num_requests":3,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":4510,"score_linear":152913.0,"score_lmetric":108615},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":34353,"ongoing_decode_tokens":34353,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":520,"cache_hit":0,"new_prefill":4510,"score_linear":34353.0,"score_lmetric":4510}],"chosen_score_linear":27639.0,"chosen_score_lmetric":4510,"t_first_token":281716.31435445,"t_first_token_unix":1779712581.3282976,"t_done":281716.736901814,"t_done_unix":1779712581.750845},{"request_id":"1283642:2:1285731:190","session_id":"1283642","input_length":14192,"t_proxy_recv":281715.759752403,"t_decision_unix":1779712580.773694,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":8192,"affinity_cache_ratio":0.5772266065388951,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":8192,"estimated_new_tokens":6000,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":11,"ongoing_decode_tokens":11,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":399,"cache_hit":0,"new_prefill":14192,"score_linear":11.0,"score_lmetric":14192},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":535,"cache_hit":8192,"new_prefill":6000,"score_linear":-8192.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":14192,"score_linear":49619.0,"score_lmetric":63811},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":14192,"score_linear":126737.0,"score_lmetric":338307},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":14192,"score_linear":27639.0,"score_lmetric":14192},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":14192,"score_linear":232709.0,"score_lmetric":266474},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":152913,"ongoing_decode_tokens":30594,"pending_prefill_tokens":31695,"num_requests":3,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":14192,"score_linear":152913.0,"score_lmetric":137661},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":34353,"ongoing_decode_tokens":34353,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":520,"cache_hit":0,"new_prefill":14192,"score_linear":34353.0,"score_lmetric":14192}],"chosen_score_linear":-8192.0,"chosen_score_lmetric":0,"t_first_token":281716.415951516,"t_first_token_unix":1779712581.429895,"t_done":281716.804469975,"t_done_unix":1779712581.818413},{"request_id":"1285558:1:1285558:188","session_id":"1285558","input_length":11,"t_proxy_recv":281715.107047307,"t_decision_unix":1779712580.1209881,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":11,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":399,"cache_hit":0,"new_prefill":11,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":535,"cache_hit":0,"new_prefill":11,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":11,"score_linear":49619.0,"score_lmetric":49630},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":11,"score_linear":126737.0,"score_lmetric":295764},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":11,"score_linear":27639.0,"score_lmetric":11},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":11,"score_linear":232709.0,"score_lmetric":238112},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":152913,"ongoing_decode_tokens":30594,"pending_prefill_tokens":31695,"num_requests":3,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":11,"score_linear":152913.0,"score_lmetric":95118},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":92395,"ongoing_decode_tokens":92395,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":507,"cache_hit":0,"new_prefill":11,"score_linear":92395.0,"score_lmetric":33}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281715.125891111,"t_first_token_unix":1779712580.1398344,"t_done":281716.840034091,"t_done_unix":1779712581.8539774},{"request_id":"1239034:4:1263738:78","session_id":"1239034","input_length":27639,"t_proxy_recv":281645.361104163,"t_decision_unix":1779712510.3750458,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":22528,"affinity_cache_ratio":0.8150801403813452,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":22528,"estimated_new_tokens":5111,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":27639,"score_linear":110048.0,"score_lmetric":137687},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":27639,"score_linear":55989.0,"score_lmetric":27639},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14329,"ongoing_decode_tokens":301,"pending_prefill_tokens":14028,"num_requests":2,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":27639,"score_linear":14329.0,"score_lmetric":83334},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":27639,"score_linear":71575.0,"score_lmetric":167862},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":258,"cache_hit":22528,"new_prefill":5111,"score_linear":-22528.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":27639,"score_linear":125063.0,"score_lmetric":290044},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":27639,"score_linear":50783.0,"score_lmetric":78422},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":0,"pending_prefill_tokens":23868,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":27639,"score_linear":45884.0,"score_lmetric":103014}],"chosen_score_linear":-22528.0,"chosen_score_lmetric":0,"t_first_token":281646.365802791,"t_first_token_unix":1779712511.3797472,"t_done":281717.71070714,"t_done_unix":1779712582.7246504},{"request_id":"1286551:1:1286551:194","session_id":"1286551","input_length":1700,"t_proxy_recv":281718.977323692,"t_decision_unix":1779712583.991265,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1700,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":399,"cache_hit":0,"new_prefill":1700,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":1700,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":1700,"score_linear":49619.0,"score_lmetric":51319},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":1700,"score_linear":126737.0,"score_lmetric":300831},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":494,"cache_hit":0,"new_prefill":1700,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":1700,"score_linear":232709.0,"score_lmetric":241490},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":152913,"ongoing_decode_tokens":30594,"pending_prefill_tokens":31695,"num_requests":3,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":1700,"score_linear":152913.0,"score_lmetric":100185},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":66251,"ongoing_decode_tokens":34353,"pending_prefill_tokens":5274,"num_requests":2,"active_p_offloads":0,"cached_blocks":520,"cache_hit":0,"new_prefill":1700,"score_linear":66251.0,"score_lmetric":13948}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281719.139842306,"t_first_token_unix":1779712584.153786,"t_done":281719.334812804,"t_done_unix":1779712584.3487556},{"request_id":"1262354:11:1284987:186","session_id":"1262354","input_length":34353,"t_proxy_recv":281713.356485777,"t_decision_unix":1779712578.3704271,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9985736325793962,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":34304,"estimated_new_tokens":49,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":399,"cache_hit":0,"new_prefill":34353,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":535,"cache_hit":0,"new_prefill":34353,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":34353,"score_linear":49619.0,"score_lmetric":83972},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":34353,"score_linear":126737.0,"score_lmetric":398790},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":34353,"score_linear":27639.0,"score_lmetric":34353},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":34353,"score_linear":232709.0,"score_lmetric":306796},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":152913,"ongoing_decode_tokens":30594,"pending_prefill_tokens":31695,"num_requests":3,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":34353,"score_linear":152913.0,"score_lmetric":198144},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":30965,"ongoing_decode_tokens":30965,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":507,"cache_hit":34304,"new_prefill":49,"score_linear":-3339.0,"score_lmetric":49}],"chosen_score_linear":-3339.0,"chosen_score_lmetric":49,"t_first_token":281713.43817399,"t_first_token_unix":1779712578.4521172,"t_done":281719.670809115,"t_done_unix":1779712584.6847522},{"request_id":"1279412:4:1286447:193","session_id":"1279412","input_length":31898,"t_proxy_recv":281718.579158889,"t_decision_unix":1779712583.5931,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":26624,"affinity_cache_ratio":0.8346604802808953,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":26624,"estimated_new_tokens":5274,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":399,"cache_hit":0,"new_prefill":31898,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":31898,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":31898,"score_linear":49619.0,"score_lmetric":81517},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":31898,"score_linear":126737.0,"score_lmetric":391425},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":494,"cache_hit":0,"new_prefill":31898,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":31898,"score_linear":232709.0,"score_lmetric":301886},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":152913,"ongoing_decode_tokens":30594,"pending_prefill_tokens":31695,"num_requests":3,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":31898,"score_linear":152913.0,"score_lmetric":190779},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":34353,"ongoing_decode_tokens":34353,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":520,"cache_hit":26624,"new_prefill":5274,"score_linear":7729.0,"score_lmetric":5274}],"chosen_score_linear":7729.0,"chosen_score_lmetric":5274,"t_first_token":281719.650716509,"t_first_token_unix":1779712584.6646597,"t_done":281721.281798714,"t_done_unix":1779712586.2957423},{"request_id":"1286804:1:1286804:195","session_id":"1286804","input_length":13962,"t_proxy_recv":281719.710543071,"t_decision_unix":1779712584.7244844,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":13962,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":399,"cache_hit":0,"new_prefill":13962,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":13962,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49619,"ongoing_decode_tokens":49619,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":13962,"score_linear":49619.0,"score_lmetric":13962},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":13962,"score_linear":126737.0,"score_lmetric":337617},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":497,"cache_hit":0,"new_prefill":13962,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":13962,"score_linear":232709.0,"score_lmetric":266014},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":152913,"ongoing_decode_tokens":30594,"pending_prefill_tokens":31695,"num_requests":3,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":13962,"score_linear":152913.0,"score_lmetric":136971},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":31898,"ongoing_decode_tokens":31898,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":521,"cache_hit":0,"new_prefill":13962,"score_linear":31898.0,"score_lmetric":13962}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281720.911790139,"t_first_token_unix":1779712585.9257336,"t_done":281721.616967196,"t_done_unix":1779712586.6309102},{"request_id":"1276136:2:1280145:162","session_id":"1276136","input_length":24519,"t_proxy_recv":281696.717988568,"t_decision_unix":1779712561.7319298,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":16896,"affinity_cache_ratio":0.6890982503364738,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":16896,"estimated_new_tokens":7623,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":24519,"score_linear":25187.0,"score_lmetric":24519},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":58752,"ongoing_decode_tokens":58752,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":24519,"score_linear":58752.0,"score_lmetric":24519},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":24519,"score_linear":150574.0,"score_lmetric":284650},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":88317,"ongoing_decode_tokens":0,"pending_prefill_tokens":88317,"num_requests":1,"active_p_offloads":0,"cached_blocks":572,"cache_hit":16896,"new_prefill":7623,"score_linear":71421.0,"score_lmetric":95940},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":465,"cache_hit":0,"new_prefill":24519,"score_linear":27639.0,"score_lmetric":24519},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":24519,"score_linear":232709.0,"score_lmetric":287128},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98626,"ongoing_decode_tokens":98626,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":518,"cache_hit":0,"new_prefill":24519,"score_linear":98626.0,"score_lmetric":73557},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44358,"ongoing_decode_tokens":44358,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":410,"cache_hit":0,"new_prefill":24519,"score_linear":44358.0,"score_lmetric":49038}],"chosen_score_linear":71421.0,"chosen_score_lmetric":95940,"t_first_token":281723.583830602,"t_first_token_unix":1779712588.5977738,"t_done":281723.994907547,"t_done_unix":1779712589.008851},{"request_id":"1287588:1:1287588:198","session_id":"1287588","input_length":9925,"t_proxy_recv":281722.260261383,"t_decision_unix":1779712587.2742023,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9925,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":426,"cache_hit":0,"new_prefill":9925,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":9925,"score_linear":77805.0,"score_lmetric":87730},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49619,"ongoing_decode_tokens":49619,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":9925,"score_linear":49619.0,"score_lmetric":9925},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":9925,"score_linear":126737.0,"score_lmetric":325506},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":497,"cache_hit":0,"new_prefill":9925,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":114612,"pending_prefill_tokens":4433,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":9925,"score_linear":232709.0,"score_lmetric":28716},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":152913,"ongoing_decode_tokens":152913,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":9925,"score_linear":152913.0,"score_lmetric":29775},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":531,"cache_hit":0,"new_prefill":9925,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281723.024165058,"t_first_token_unix":1779712588.0381083,"t_done":281724.346346625,"t_done_unix":1779712589.36029},{"request_id":"1268831:5:1284640:184","session_id":"1268831","input_length":42264,"t_proxy_recv":281711.954234172,"t_decision_unix":1779712576.9681756,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":32256,"affinity_cache_ratio":0.7632027257240205,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":32256,"estimated_new_tokens":10008,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":399,"cache_hit":0,"new_prefill":42264,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":535,"cache_hit":0,"new_prefill":42264,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":42264,"score_linear":49619.0,"score_lmetric":91883},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":42264,"score_linear":126737.0,"score_lmetric":422523},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":42264,"score_linear":27639.0,"score_lmetric":42264},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":42264,"score_linear":232709.0,"score_lmetric":322618},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":110649,"ongoing_decode_tokens":30594,"pending_prefill_tokens":21687,"num_requests":2,"active_p_offloads":0,"cached_blocks":597,"cache_hit":32256,"new_prefill":10008,"score_linear":78393.0,"score_lmetric":63390},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":30965,"ongoing_decode_tokens":30965,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":507,"cache_hit":0,"new_prefill":42264,"score_linear":30965.0,"score_lmetric":42264}],"chosen_score_linear":78393.0,"chosen_score_lmetric":63390,"t_first_token":281721.770160781,"t_first_token_unix":1779712586.7841074,"t_done":281724.568752189,"t_done_unix":1779712589.5826957},{"request_id":"1288143:1:1288143:200","session_id":"1288143","input_length":448,"t_proxy_recv":281724.224714339,"t_decision_unix":1779712589.2386558,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":448,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":426,"cache_hit":0,"new_prefill":448,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":448,"score_linear":77805.0,"score_lmetric":78253},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49619,"ongoing_decode_tokens":49619,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":448,"score_linear":49619.0,"score_lmetric":448},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":128141,"ongoing_decode_tokens":102218,"pending_prefill_tokens":1859,"num_requests":3,"active_p_offloads":0,"cached_blocks":586,"cache_hit":0,"new_prefill":448,"score_linear":128141.0,"score_lmetric":6921},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":497,"cache_hit":0,"new_prefill":448,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":114612,"pending_prefill_tokens":4433,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":448,"score_linear":232709.0,"score_lmetric":9762},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":152913,"ongoing_decode_tokens":152913,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":448,"score_linear":152913.0,"score_lmetric":1344},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":9925,"ongoing_decode_tokens":9925,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":531,"cache_hit":0,"new_prefill":448,"score_linear":9925.0,"score_lmetric":448}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281724.270051026,"t_first_token_unix":1779712589.2839942,"t_done":281724.792291592,"t_done_unix":1779712589.8062346},{"request_id":"1269373:5:1282812:174","session_id":"1269373","input_length":30594,"t_proxy_recv":281707.2063286,"t_decision_unix":1779712572.2202704,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":30208,"affinity_cache_ratio":0.9873831470222919,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":30208,"estimated_new_tokens":386,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":4096,"new_prefill":26498,"score_linear":21091.0,"score_lmetric":26498},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":30594,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16299,"ongoing_decode_tokens":0,"pending_prefill_tokens":16299,"num_requests":1,"active_p_offloads":0,"cached_blocks":566,"cache_hit":0,"new_prefill":30594,"score_linear":16299.0,"score_lmetric":46893},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":7680,"new_prefill":22914,"score_linear":119057.0,"score_lmetric":364473},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":30594,"score_linear":27639.0,"score_lmetric":30594},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":30594,"score_linear":232709.0,"score_lmetric":299278},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58032,"ongoing_decode_tokens":58032,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":580,"cache_hit":30208,"new_prefill":386,"score_linear":27824.0,"score_lmetric":386},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":26426,"ongoing_decode_tokens":26426,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":493,"cache_hit":7680,"new_prefill":22914,"score_linear":18746.0,"score_lmetric":22914}],"chosen_score_linear":27824.0,"chosen_score_lmetric":386,"t_first_token":281707.30229292,"t_first_token_unix":1779712572.316236,"t_done":281724.834422579,"t_done_unix":1779712589.8483658},{"request_id":"1279994:1:1279994:161","session_id":"1279994","input_length":88317,"t_proxy_recv":281696.295378881,"t_decision_unix":1779712561.30932,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":88317,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":88317,"score_linear":25187.0,"score_lmetric":88317},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":58752,"ongoing_decode_tokens":58752,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":88317,"score_linear":58752.0,"score_lmetric":88317},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":88317,"score_linear":150574.0,"score_lmetric":412246},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":88317,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":465,"cache_hit":0,"new_prefill":88317,"score_linear":27639.0,"score_lmetric":88317},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":88317,"score_linear":232709.0,"score_lmetric":414724},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98626,"ongoing_decode_tokens":98626,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":518,"cache_hit":0,"new_prefill":88317,"score_linear":98626.0,"score_lmetric":264951},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44358,"ongoing_decode_tokens":44358,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":410,"cache_hit":0,"new_prefill":88317,"score_linear":44358.0,"score_lmetric":176634}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281722.490610513,"t_first_token_unix":1779712587.5045555,"t_done":281725.024557874,"t_done_unix":1779712590.038502},{"request_id":"1276136:3:1280987:166","session_id":"1276136","input_length":25923,"t_proxy_recv":281724.001624138,"t_decision_unix":1779712589.0155656,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9282876210315164,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":1859,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":426,"cache_hit":0,"new_prefill":25923,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":25923,"score_linear":77805.0,"score_lmetric":103728},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49619,"ongoing_decode_tokens":49619,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":25923,"score_linear":49619.0,"score_lmetric":25923},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":102218,"ongoing_decode_tokens":102218,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":586,"cache_hit":24064,"new_prefill":1859,"score_linear":78154.0,"score_lmetric":3718},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":497,"cache_hit":0,"new_prefill":25923,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":114612,"pending_prefill_tokens":4433,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":25923,"score_linear":232709.0,"score_lmetric":60712},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":152913,"ongoing_decode_tokens":152913,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":25923,"score_linear":152913.0,"score_lmetric":77769},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":9925,"ongoing_decode_tokens":9925,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":531,"cache_hit":0,"new_prefill":25923,"score_linear":9925.0,"score_lmetric":25923}],"chosen_score_linear":78154.0,"chosen_score_lmetric":3718,"t_first_token":281724.343330788,"t_first_token_unix":1779712589.357274,"t_done":281725.716490197,"t_done_unix":1779712590.7304506},{"request_id":"1284132:1:1284132:182","session_id":"1284132","input_length":49619,"t_proxy_recv":281710.231308571,"t_decision_unix":1779712575.2452497,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":49619,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":399,"cache_hit":0,"new_prefill":49619,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":535,"cache_hit":0,"new_prefill":49619,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":49619,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":49619,"score_linear":126737.0,"score_lmetric":444588},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":49619,"score_linear":27639.0,"score_lmetric":49619},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":49619,"score_linear":232709.0,"score_lmetric":337328},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":110649,"ongoing_decode_tokens":30594,"pending_prefill_tokens":21687,"num_requests":2,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":49619,"score_linear":110649.0,"score_lmetric":142612},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":91673,"ongoing_decode_tokens":91673,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":493,"cache_hit":0,"new_prefill":49619,"score_linear":91673.0,"score_lmetric":148857}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281719.483380248,"t_first_token_unix":1779712584.497324,"t_done":281726.210146844,"t_done_unix":1779712591.2240903},{"request_id":"1279994:2:1283011:177","session_id":"1279994","input_length":88940,"t_proxy_recv":281725.040470713,"t_decision_unix":1779712590.0544121,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":88064,"affinity_cache_ratio":0.990150663368563,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":88064,"estimated_new_tokens":876,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":427,"cache_hit":0,"new_prefill":88940,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":88940,"score_linear":77805.0,"score_lmetric":166745},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49619,"ongoing_decode_tokens":49619,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":88940,"score_linear":49619.0,"score_lmetric":88940},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":39824,"ongoing_decode_tokens":39824,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":758,"cache_hit":88064,"new_prefill":876,"score_linear":-48240.0,"score_lmetric":1752},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":497,"cache_hit":0,"new_prefill":88940,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":88940,"score_linear":232709.0,"score_lmetric":177880},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":118843,"ongoing_decode_tokens":80055,"pending_prefill_tokens":8068,"num_requests":2,"active_p_offloads":0,"cached_blocks":617,"cache_hit":0,"new_prefill":88940,"score_linear":118843.0,"score_lmetric":194016},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":550,"cache_hit":0,"new_prefill":88940,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-48240.0,"chosen_score_lmetric":1752,"t_first_token":281725.507647922,"t_first_token_unix":1779712590.5215912,"t_done":281726.451380926,"t_done_unix":1779712591.4653244},{"request_id":"1272313:4:1281720:170","session_id":"1272313","input_length":13901,"t_proxy_recv":281702.01935951,"t_decision_unix":1779712567.0333009,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":11264,"affinity_cache_ratio":0.8103014171642328,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":2637,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":4096,"new_prefill":9805,"score_linear":21091.0,"score_lmetric":9805},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":501,"cache_hit":0,"new_prefill":13901,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":13901,"score_linear":150574.0,"score_lmetric":263414},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":112836,"ongoing_decode_tokens":0,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":572,"cache_hit":11264,"new_prefill":2637,"score_linear":101572.0,"score_lmetric":197154},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":466,"cache_hit":0,"new_prefill":13901,"score_linear":27639.0,"score_lmetric":13901},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":13901,"score_linear":232709.0,"score_lmetric":265892},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":88294,"ongoing_decode_tokens":30262,"pending_prefill_tokens":8368,"num_requests":2,"active_p_offloads":0,"cached_blocks":555,"cache_hit":10752,"new_prefill":3149,"score_linear":77542.0,"score_lmetric":23034},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60454,"ongoing_decode_tokens":60454,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":472,"cache_hit":7680,"new_prefill":6221,"score_linear":52774.0,"score_lmetric":12442}],"chosen_score_linear":101572.0,"chosen_score_lmetric":197154,"t_first_token":281723.586412447,"t_first_token_unix":1779712588.6003559,"t_done":281727.544829125,"t_done_unix":1779712592.5587726},{"request_id":"1286551:2:1289450:202","session_id":"1286551","input_length":9610,"t_proxy_recv":281728.570214641,"t_decision_unix":1779712593.5841553,"policy":"unified","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":1536,"affinity_cache_ratio":0.1598335067637877,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":8074,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":427,"cache_hit":0,"new_prefill":9610,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":9610,"score_linear":77805.0,"score_lmetric":87415},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":9610,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":104111,"ongoing_decode_tokens":104111,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":767,"cache_hit":0,"new_prefill":9610,"score_linear":104111.0,"score_lmetric":19220},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":497,"cache_hit":1536,"new_prefill":8074,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":9610,"score_linear":232709.0,"score_lmetric":19220},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":118843,"ongoing_decode_tokens":118843,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":617,"cache_hit":0,"new_prefill":9610,"score_linear":118843.0,"score_lmetric":19220},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44920,"ongoing_decode_tokens":0,"pending_prefill_tokens":13688,"num_requests":1,"active_p_offloads":0,"cached_blocks":550,"cache_hit":0,"new_prefill":9610,"score_linear":44920.0,"score_lmetric":23298}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":281729.201049347,"t_first_token_unix":1779712594.214993,"t_done":281729.570844471,"t_done_unix":1779712594.5847876},{"request_id":"1266668:6:1284003:181","session_id":"1266668","input_length":80055,"t_proxy_recv":281709.773870458,"t_decision_unix":1779712574.7878118,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":58368,"affinity_cache_ratio":0.7290987446130786,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":58368,"estimated_new_tokens":21687,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":399,"cache_hit":4096,"new_prefill":75959,"score_linear":-4096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":535,"cache_hit":0,"new_prefill":80055,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":80055,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":7680,"new_prefill":72375,"score_linear":119057.0,"score_lmetric":512856},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":80055,"score_linear":27639.0,"score_lmetric":80055},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":80055,"score_linear":232709.0,"score_lmetric":398200},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":30594,"ongoing_decode_tokens":30594,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":58368,"new_prefill":21687,"score_linear":-27774.0,"score_lmetric":21687},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":91673,"ongoing_decode_tokens":26426,"pending_prefill_tokens":6879,"num_requests":3,"active_p_offloads":0,"cached_blocks":493,"cache_hit":7680,"new_prefill":72375,"score_linear":83993.0,"score_lmetric":237762}],"chosen_score_linear":-27774.0,"chosen_score_lmetric":21687,"t_first_token":281720.033883581,"t_first_token_unix":1779712585.047827,"t_done":281730.225393601,"t_done_unix":1779712595.2393372},{"request_id":"1287588:2:1289701:205","session_id":"1287588","input_length":10195,"t_proxy_recv":281729.488444919,"t_decision_unix":1779712594.502386,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":9728,"affinity_cache_ratio":0.954193231976459,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":9728,"estimated_new_tokens":467,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":427,"cache_hit":0,"new_prefill":10195,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":10195,"score_linear":77805.0,"score_lmetric":88000},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":10195,"score_linear":82014.0,"score_lmetric":92209},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":104111,"ongoing_decode_tokens":104111,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":767,"cache_hit":0,"new_prefill":10195,"score_linear":104111.0,"score_lmetric":20390},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":9610,"ongoing_decode_tokens":9610,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":497,"cache_hit":0,"new_prefill":10195,"score_linear":9610.0,"score_lmetric":10195},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":10195,"score_linear":232709.0,"score_lmetric":20390},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":118843,"ongoing_decode_tokens":118843,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":617,"cache_hit":0,"new_prefill":10195,"score_linear":118843.0,"score_lmetric":20390},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":77111,"ongoing_decode_tokens":0,"pending_prefill_tokens":14135,"num_requests":2,"active_p_offloads":0,"cached_blocks":550,"cache_hit":9728,"new_prefill":467,"score_linear":67383.0,"score_lmetric":29204}],"chosen_score_linear":67383.0,"chosen_score_lmetric":29204,"t_first_token":281731.276111698,"t_first_token_unix":1779712596.2900548,"t_done":281731.396555599,"t_done_unix":1779712596.410499},{"request_id":"1290442:1:1290442:210","session_id":"1290442","input_length":7261,"t_proxy_recv":281731.841522308,"t_decision_unix":1779712596.855464,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":14522,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7261,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":23228,"ongoing_decode_tokens":0,"pending_prefill_tokens":23228,"num_requests":1,"active_p_offloads":0,"cached_blocks":427,"cache_hit":0,"new_prefill":7261,"score_linear":23228.0,"score_lmetric":30489},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":7261,"score_linear":77805.0,"score_lmetric":85066},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":7261,"score_linear":82014.0,"score_lmetric":89275},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":104111,"ongoing_decode_tokens":104111,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":767,"cache_hit":0,"new_prefill":7261,"score_linear":104111.0,"score_lmetric":14522},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":13552,"ongoing_decode_tokens":0,"pending_prefill_tokens":12016,"num_requests":1,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":7261,"score_linear":13552.0,"score_lmetric":19277},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":7261,"score_linear":232709.0,"score_lmetric":14522},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":126529,"ongoing_decode_tokens":38788,"pending_prefill_tokens":7357,"num_requests":2,"active_p_offloads":0,"cached_blocks":660,"cache_hit":0,"new_prefill":7261,"score_linear":126529.0,"score_lmetric":29236},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":77111,"ongoing_decode_tokens":77111,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":550,"cache_hit":0,"new_prefill":7261,"score_linear":77111.0,"score_lmetric":14522}],"chosen_score_linear":104111.0,"chosen_score_lmetric":14522,"t_first_token":281732.3507485,"t_first_token_unix":1779712597.3646924,"t_done":281732.74527412,"t_done_unix":1779712597.7592175},{"request_id":"1279412:4:1289482:203","session_id":"1279412","input_length":32191,"t_proxy_recv":281728.649927459,"t_decision_unix":1779712593.663869,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":31744,"affinity_cache_ratio":0.9861141312789289,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":31744,"estimated_new_tokens":447,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":427,"cache_hit":0,"new_prefill":32191,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":32191,"score_linear":77805.0,"score_lmetric":109996},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":32191,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":104111,"ongoing_decode_tokens":104111,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":767,"cache_hit":0,"new_prefill":32191,"score_linear":104111.0,"score_lmetric":64382},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":9610,"ongoing_decode_tokens":0,"pending_prefill_tokens":8074,"num_requests":1,"active_p_offloads":0,"cached_blocks":497,"cache_hit":0,"new_prefill":32191,"score_linear":9610.0,"score_lmetric":40265},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":32191,"score_linear":232709.0,"score_lmetric":64382},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":118843,"ongoing_decode_tokens":118843,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":617,"cache_hit":0,"new_prefill":32191,"score_linear":118843.0,"score_lmetric":64382},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44920,"ongoing_decode_tokens":0,"pending_prefill_tokens":13688,"num_requests":1,"active_p_offloads":0,"cached_blocks":550,"cache_hit":31744,"new_prefill":447,"score_linear":13176.0,"score_lmetric":14135}],"chosen_score_linear":13176.0,"chosen_score_lmetric":14135,"t_first_token":281731.272707901,"t_first_token_unix":1779712596.2866514,"t_done":281732.802367557,"t_done_unix":1779712597.8163114},{"request_id":"1279994:3:1286181:192","session_id":"1279994","input_length":89226,"t_proxy_recv":281726.471542476,"t_decision_unix":1779712591.4854841,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":88576,"affinity_cache_ratio":0.9927151278775245,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":88576,"estimated_new_tokens":650,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":427,"cache_hit":0,"new_prefill":89226,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":89226,"score_linear":77805.0,"score_lmetric":167031},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":89226,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":13901,"ongoing_decode_tokens":13901,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":762,"cache_hit":88576,"new_prefill":650,"score_linear":-74675.0,"score_lmetric":650},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":497,"cache_hit":0,"new_prefill":89226,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":89226,"score_linear":232709.0,"score_lmetric":178452},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":118843,"ongoing_decode_tokens":80055,"pending_prefill_tokens":8068,"num_requests":2,"active_p_offloads":0,"cached_blocks":617,"cache_hit":0,"new_prefill":89226,"score_linear":118843.0,"score_lmetric":194588},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":550,"cache_hit":0,"new_prefill":89226,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-74675.0,"chosen_score_lmetric":650,"t_first_token":281726.758225526,"t_first_token_unix":1779712591.7721686,"t_done":281733.058660727,"t_done_unix":1779712598.0726042},{"request_id":"1290426:1:1290426:209","session_id":"1290426","input_length":13552,"t_proxy_recv":281731.816515331,"t_decision_unix":1779712596.8304567,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":12016,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":23228,"ongoing_decode_tokens":0,"pending_prefill_tokens":23228,"num_requests":1,"active_p_offloads":0,"cached_blocks":427,"cache_hit":0,"new_prefill":13552,"score_linear":23228.0,"score_lmetric":36780},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":13552,"score_linear":77805.0,"score_lmetric":91357},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":13552,"score_linear":82014.0,"score_lmetric":95566},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":104111,"ongoing_decode_tokens":104111,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":767,"cache_hit":0,"new_prefill":13552,"score_linear":104111.0,"score_lmetric":27104},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":512,"cache_hit":1536,"new_prefill":12016,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":13552,"score_linear":232709.0,"score_lmetric":27104},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":126529,"ongoing_decode_tokens":38788,"pending_prefill_tokens":7357,"num_requests":2,"active_p_offloads":0,"cached_blocks":660,"cache_hit":0,"new_prefill":13552,"score_linear":126529.0,"score_lmetric":41818},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":77111,"ongoing_decode_tokens":77111,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":550,"cache_hit":0,"new_prefill":13552,"score_linear":77111.0,"score_lmetric":27104}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":281732.907592838,"t_first_token_unix":1779712597.9215367,"t_done":281733.286290855,"t_done_unix":1779712598.300234},{"request_id":"1272313:5:1284744:185","session_id":"1272313","input_length":14885,"t_proxy_recv":281727.5490995,"t_decision_unix":1779712592.563041,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":13824,"affinity_cache_ratio":0.9287201881088344,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":13824,"estimated_new_tokens":1061,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":427,"cache_hit":4096,"new_prefill":10789,"score_linear":-4096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":14885,"score_linear":77805.0,"score_lmetric":92690},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":14885,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":89226,"ongoing_decode_tokens":89226,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":767,"cache_hit":13824,"new_prefill":1061,"score_linear":75402.0,"score_lmetric":1061},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":497,"cache_hit":0,"new_prefill":14885,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":14885,"score_linear":232709.0,"score_lmetric":29770},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":118843,"ongoing_decode_tokens":118843,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":617,"cache_hit":10752,"new_prefill":4133,"score_linear":108091.0,"score_lmetric":8266},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":550,"cache_hit":7680,"new_prefill":7205,"score_linear":-7680.0,"score_lmetric":0}],"chosen_score_linear":75402.0,"chosen_score_lmetric":1061,"t_first_token":281727.789037667,"t_first_token_unix":1779712592.8029816,"t_done":281734.080856487,"t_done_unix":1779712599.0947998},{"request_id":"1291100:1:1291100:212","session_id":"1291100","input_length":1744,"t_proxy_recv":281734.056632825,"t_decision_unix":1779712599.070574,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":1744,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":23228,"ongoing_decode_tokens":0,"pending_prefill_tokens":23228,"num_requests":1,"active_p_offloads":0,"cached_blocks":427,"cache_hit":0,"new_prefill":1744,"score_linear":23228.0,"score_lmetric":24972},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":1744,"score_linear":77805.0,"score_lmetric":79549},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":1744,"score_linear":82014.0,"score_lmetric":83758},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":104634,"ongoing_decode_tokens":104634,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":783,"cache_hit":0,"new_prefill":1744,"score_linear":104634.0,"score_lmetric":3488},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":521,"cache_hit":0,"new_prefill":1744,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":1744,"score_linear":232709.0,"score_lmetric":3488},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":126529,"ongoing_decode_tokens":38788,"pending_prefill_tokens":7357,"num_requests":2,"active_p_offloads":0,"cached_blocks":660,"cache_hit":0,"new_prefill":1744,"score_linear":126529.0,"score_lmetric":18202},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":551,"cache_hit":0,"new_prefill":1744,"score_linear":44920.0,"score_lmetric":1744}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281734.171463389,"t_first_token_unix":1779712599.1854067,"t_done":281734.569057803,"t_done_unix":1779712599.5830011},{"request_id":"1290365:1:1290365:208","session_id":"1290365","input_length":23228,"t_proxy_recv":281731.636395699,"t_decision_unix":1779712596.650337,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":23228,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":427,"cache_hit":0,"new_prefill":23228,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":23228,"score_linear":77805.0,"score_lmetric":101033},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":23228,"score_linear":82014.0,"score_lmetric":105242},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":104111,"ongoing_decode_tokens":104111,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":767,"cache_hit":0,"new_prefill":23228,"score_linear":104111.0,"score_lmetric":46456},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":23228,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":23228,"score_linear":232709.0,"score_lmetric":46456},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":126529,"ongoing_decode_tokens":38788,"pending_prefill_tokens":7357,"num_requests":2,"active_p_offloads":0,"cached_blocks":660,"cache_hit":0,"new_prefill":23228,"score_linear":126529.0,"score_lmetric":61170},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":77111,"ongoing_decode_tokens":77111,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":550,"cache_hit":0,"new_prefill":23228,"score_linear":77111.0,"score_lmetric":46456}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281734.215371626,"t_first_token_unix":1779712599.2293148,"t_done":281734.947126983,"t_done_unix":1779712599.9610705},{"request_id":"1279994:4:1290711:211","session_id":"1279994","input_length":89749,"t_proxy_recv":281733.076941434,"t_decision_unix":1779712598.090883,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":89600,"affinity_cache_ratio":0.9983398143711908,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":89600,"estimated_new_tokens":149,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":23228,"ongoing_decode_tokens":0,"pending_prefill_tokens":23228,"num_requests":1,"active_p_offloads":0,"cached_blocks":427,"cache_hit":0,"new_prefill":89749,"score_linear":23228.0,"score_lmetric":112977},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":89749,"score_linear":77805.0,"score_lmetric":167554},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":89749,"score_linear":82014.0,"score_lmetric":171763},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":14885,"ongoing_decode_tokens":14885,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":783,"cache_hit":89600,"new_prefill":149,"score_linear":-74715.0,"score_lmetric":149},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":13552,"ongoing_decode_tokens":13552,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":89749,"score_linear":13552.0,"score_lmetric":89749},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":89749,"score_linear":232709.0,"score_lmetric":179498},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":126529,"ongoing_decode_tokens":38788,"pending_prefill_tokens":7357,"num_requests":2,"active_p_offloads":0,"cached_blocks":660,"cache_hit":0,"new_prefill":89749,"score_linear":126529.0,"score_lmetric":194212},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":551,"cache_hit":0,"new_prefill":89749,"score_linear":44920.0,"score_lmetric":89749}],"chosen_score_linear":-74715.0,"chosen_score_lmetric":149,"t_first_token":281733.257174002,"t_first_token_unix":1779712598.271117,"t_done":281735.653851439,"t_done_unix":1779712600.6677947},{"request_id":"1269373:6:1285668:189","session_id":"1269373","input_length":38788,"t_proxy_recv":281724.845468743,"t_decision_unix":1779712589.8594103,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":30720,"affinity_cache_ratio":0.7919975250077343,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":30720,"estimated_new_tokens":8068,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":427,"cache_hit":4096,"new_prefill":34692,"score_linear":-4096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":38788,"score_linear":77805.0,"score_lmetric":116593},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49619,"ongoing_decode_tokens":49619,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":38788,"score_linear":49619.0,"score_lmetric":38788},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":128141,"ongoing_decode_tokens":128141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":586,"cache_hit":7680,"new_prefill":31108,"score_linear":120461.0,"score_lmetric":93324},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":497,"cache_hit":0,"new_prefill":38788,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":38788,"score_linear":232709.0,"score_lmetric":77576},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80055,"ongoing_decode_tokens":80055,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":617,"cache_hit":30720,"new_prefill":8068,"score_linear":49335.0,"score_lmetric":8068},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":550,"cache_hit":7680,"new_prefill":31108,"score_linear":-7680.0,"score_lmetric":0}],"chosen_score_linear":49335.0,"chosen_score_lmetric":8068,"t_first_token":281726.754046106,"t_first_token_unix":1779712591.7679892,"t_done":281735.996516169,"t_done_unix":1779712601.0104594},{"request_id":"1258859:3:1278118:155","session_id":"1258859","input_length":118097,"t_proxy_recv":281689.936872262,"t_decision_unix":1779712554.9508135,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":113664,"affinity_cache_ratio":0.962463060026927,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":113664,"estimated_new_tokens":4433,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34071,"ongoing_decode_tokens":34071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":118097,"score_linear":34071.0,"score_lmetric":236194},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":118097,"score_linear":120167.0,"score_lmetric":448880},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":118097,"score_linear":150574.0,"score_lmetric":471806},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49794,"ongoing_decode_tokens":49794,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":530,"cache_hit":0,"new_prefill":118097,"score_linear":49794.0,"score_lmetric":236194},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":118097,"score_linear":121069.0,"score_lmetric":423054},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":321,"cache_hit":113664,"new_prefill":4433,"score_linear":948.0,"score_lmetric":119045},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106829,"ongoing_decode_tokens":58084,"pending_prefill_tokens":14441,"num_requests":4,"active_p_offloads":0,"cached_blocks":462,"cache_hit":0,"new_prefill":118097,"score_linear":106829.0,"score_lmetric":530152},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":55652,"ongoing_decode_tokens":0,"pending_prefill_tokens":24420,"num_requests":2,"active_p_offloads":0,"cached_blocks":410,"cache_hit":0,"new_prefill":118097,"score_linear":55652.0,"score_lmetric":285034}],"chosen_score_linear":948.0,"chosen_score_lmetric":119045,"t_first_token":281724.77397246,"t_first_token_unix":1779712589.787916,"t_done":281736.16875855,"t_done_unix":1779712601.182702},{"request_id":"1272313:6:1287745:199","session_id":"1272313","input_length":15547,"t_proxy_recv":281734.086706134,"t_decision_unix":1779712599.1006477,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":15360,"affinity_cache_ratio":0.9879719560043738,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":15360,"estimated_new_tokens":187,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":23228,"ongoing_decode_tokens":0,"pending_prefill_tokens":23228,"num_requests":1,"active_p_offloads":0,"cached_blocks":427,"cache_hit":4096,"new_prefill":11451,"score_linear":19132.0,"score_lmetric":34679},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":15547,"score_linear":77805.0,"score_lmetric":93352},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":15547,"score_linear":82014.0,"score_lmetric":97561},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":89749,"ongoing_decode_tokens":89749,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":786,"cache_hit":15360,"new_prefill":187,"score_linear":74389.0,"score_lmetric":187},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":1744,"ongoing_decode_tokens":0,"pending_prefill_tokens":1744,"num_requests":1,"active_p_offloads":0,"cached_blocks":521,"cache_hit":0,"new_prefill":15547,"score_linear":1744.0,"score_lmetric":17291},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":15547,"score_linear":232709.0,"score_lmetric":31094},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":126529,"ongoing_decode_tokens":38788,"pending_prefill_tokens":7357,"num_requests":2,"active_p_offloads":0,"cached_blocks":660,"cache_hit":10752,"new_prefill":4795,"score_linear":115777.0,"score_lmetric":24304},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":551,"cache_hit":7680,"new_prefill":7867,"score_linear":37240.0,"score_lmetric":7867}],"chosen_score_linear":74389.0,"chosen_score_lmetric":187,"t_first_token":281734.169191265,"t_first_token_unix":1779712599.1831346,"t_done":281736.492178544,"t_done_unix":1779712601.5061224},{"request_id":"1262354:12:1291548:213","session_id":"1262354","input_length":35099,"t_proxy_recv":281735.699161666,"t_decision_unix":1779712600.7131028,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9919370922248497,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":283,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":472,"cache_hit":0,"new_prefill":35099,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":35099,"score_linear":77805.0,"score_lmetric":112904},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":35099,"score_linear":82014.0,"score_lmetric":117113},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15547,"ongoing_decode_tokens":15547,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":786,"cache_hit":0,"new_prefill":35099,"score_linear":15547.0,"score_lmetric":35099},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":524,"cache_hit":0,"new_prefill":35099,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":35099,"score_linear":232709.0,"score_lmetric":70198},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":126529,"ongoing_decode_tokens":126529,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":660,"cache_hit":0,"new_prefill":35099,"score_linear":126529.0,"score_lmetric":70198},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":551,"cache_hit":34816,"new_prefill":283,"score_linear":10104.0,"score_lmetric":283}],"chosen_score_linear":10104.0,"chosen_score_lmetric":283,"t_first_token":281735.782582007,"t_first_token_unix":1779712600.7965252,"t_done":281736.927726441,"t_done_unix":1779712601.94167},{"request_id":"1279412:5:1291651:214","session_id":"1279412","input_length":32359,"t_proxy_recv":281735.98367438,"t_decision_unix":1779712600.9976156,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":32256,"affinity_cache_ratio":0.9968169597329954,"affinity_num_requests":2,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":32256,"estimated_new_tokens":103,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":472,"cache_hit":0,"new_prefill":32359,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":32359,"score_linear":77805.0,"score_lmetric":110164},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":32359,"score_linear":82014.0,"score_lmetric":114373},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15547,"ongoing_decode_tokens":15547,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":786,"cache_hit":0,"new_prefill":32359,"score_linear":15547.0,"score_lmetric":32359},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":524,"cache_hit":0,"new_prefill":32359,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":32359,"score_linear":232709.0,"score_lmetric":64718},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":126529,"ongoing_decode_tokens":126529,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":660,"cache_hit":0,"new_prefill":32359,"score_linear":126529.0,"score_lmetric":64718},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80019,"ongoing_decode_tokens":80019,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":551,"cache_hit":32256,"new_prefill":103,"score_linear":47763.0,"score_lmetric":206}],"chosen_score_linear":47763.0,"chosen_score_lmetric":206,"t_first_token":281736.057980781,"t_first_token_unix":1779712601.071924,"t_done":281737.266261536,"t_done_unix":1779712602.2802048},{"request_id":"1291944:1:1291944:216","session_id":"1291944","input_length":8144,"t_proxy_recv":281737.038120069,"t_decision_unix":1779712602.052061,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":8144,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":42153,"ongoing_decode_tokens":0,"pending_prefill_tokens":42153,"num_requests":1,"active_p_offloads":0,"cached_blocks":472,"cache_hit":0,"new_prefill":8144,"score_linear":42153.0,"score_lmetric":50297},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":8144,"score_linear":77805.0,"score_lmetric":85949},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":8144,"score_linear":82014.0,"score_lmetric":90158},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15969,"ongoing_decode_tokens":15969,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":786,"cache_hit":0,"new_prefill":8144,"score_linear":15969.0,"score_lmetric":8144},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":524,"cache_hit":0,"new_prefill":8144,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":233348,"ongoing_decode_tokens":233348,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":330,"cache_hit":0,"new_prefill":8144,"score_linear":233348.0,"score_lmetric":16288},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":130486,"ongoing_decode_tokens":87741,"pending_prefill_tokens":3833,"num_requests":2,"active_p_offloads":0,"cached_blocks":676,"cache_hit":0,"new_prefill":8144,"score_linear":130486.0,"score_lmetric":23954},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":77279,"ongoing_decode_tokens":77279,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":551,"cache_hit":0,"new_prefill":8144,"score_linear":77279.0,"score_lmetric":16288}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281737.609432116,"t_first_token_unix":1779712602.623376,"t_done":281737.754209719,"t_done_unix":1779712602.7681527},{"request_id":"1272313:7:1290281:206","session_id":"1272313","input_length":15969,"t_proxy_recv":281736.496964519,"t_decision_unix":1779712601.5109055,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":15360,"affinity_cache_ratio":0.96186361074582,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":15360,"estimated_new_tokens":609,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":472,"cache_hit":4096,"new_prefill":11873,"score_linear":-4096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":15969,"score_linear":77805.0,"score_lmetric":93774},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":15969,"score_linear":82014.0,"score_lmetric":97983},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":786,"cache_hit":15360,"new_prefill":609,"score_linear":-15360.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":524,"cache_hit":0,"new_prefill":15969,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":233348,"ongoing_decode_tokens":233348,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":330,"cache_hit":0,"new_prefill":15969,"score_linear":233348.0,"score_lmetric":31938},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":130486,"ongoing_decode_tokens":87741,"pending_prefill_tokens":3833,"num_requests":2,"active_p_offloads":0,"cached_blocks":676,"cache_hit":10752,"new_prefill":5217,"score_linear":119734.0,"score_lmetric":18100},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":112378,"ongoing_decode_tokens":112378,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":551,"cache_hit":7680,"new_prefill":8289,"score_linear":104698.0,"score_lmetric":24867}],"chosen_score_linear":-15360.0,"chosen_score_lmetric":0,"t_first_token":281736.570676586,"t_first_token_unix":1779712601.5846205,"t_done":281738.563283421,"t_done_unix":1779712603.577227},{"request_id":"1286551:3:1292589:219","session_id":"1286551","input_length":16914,"t_proxy_recv":281738.905872607,"t_decision_unix":1779712603.919814,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":9216,"affinity_cache_ratio":0.54487406881873,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":9216,"estimated_new_tokens":7698,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":65567,"ongoing_decode_tokens":0,"pending_prefill_tokens":42527,"num_requests":2,"active_p_offloads":0,"cached_blocks":472,"cache_hit":0,"new_prefill":16914,"score_linear":65567.0,"score_lmetric":118882},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":16914,"score_linear":77805.0,"score_lmetric":94719},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":16914,"score_linear":82014.0,"score_lmetric":98928},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":787,"cache_hit":0,"new_prefill":16914,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":539,"cache_hit":9216,"new_prefill":7698,"score_linear":-9216.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":233348,"ongoing_decode_tokens":233348,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":330,"cache_hit":0,"new_prefill":16914,"score_linear":233348.0,"score_lmetric":33828},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":130486,"ongoing_decode_tokens":130486,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":676,"cache_hit":0,"new_prefill":16914,"score_linear":130486.0,"score_lmetric":33828},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":551,"cache_hit":0,"new_prefill":16914,"score_linear":44920.0,"score_lmetric":16914}],"chosen_score_linear":-9216.0,"chosen_score_lmetric":0,"t_first_token":281739.804272267,"t_first_token_unix":1779712604.8182166,"t_done":281739.985312697,"t_done_unix":1779712604.9992564},{"request_id":"1266668:7:1287468:197","session_id":"1266668","input_length":87741,"t_proxy_recv":281730.242775957,"t_decision_unix":1779712595.2567172,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":80384,"affinity_cache_ratio":0.916150944256391,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":80384,"estimated_new_tokens":7357,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":427,"cache_hit":4096,"new_prefill":83645,"score_linear":-4096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":87741,"score_linear":77805.0,"score_lmetric":165546},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":87741,"score_linear":82014.0,"score_lmetric":169755},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":104111,"ongoing_decode_tokens":104111,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":767,"cache_hit":7680,"new_prefill":80061,"score_linear":96431.0,"score_lmetric":160122},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":87741,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":87741,"score_linear":232709.0,"score_lmetric":175482},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":38788,"ongoing_decode_tokens":38788,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":660,"cache_hit":80384,"new_prefill":7357,"score_linear":-41596.0,"score_lmetric":7357},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":87306,"ongoing_decode_tokens":0,"pending_prefill_tokens":14602,"num_requests":3,"active_p_offloads":0,"cached_blocks":550,"cache_hit":7680,"new_prefill":80061,"score_linear":79626.0,"score_lmetric":283989}],"chosen_score_linear":-41596.0,"chosen_score_lmetric":7357,"t_first_token":281734.122754795,"t_first_token_unix":1779712599.1366985,"t_done":281740.219357687,"t_done_unix":1779712605.2333012},{"request_id":"1292750:1:1292750:220","session_id":"1292750","input_length":4684,"t_proxy_recv":281739.537349006,"t_decision_unix":1779712604.5512903,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":4684,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":65567,"ongoing_decode_tokens":0,"pending_prefill_tokens":42527,"num_requests":2,"active_p_offloads":0,"cached_blocks":472,"cache_hit":0,"new_prefill":4684,"score_linear":65567.0,"score_lmetric":94422},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":4684,"score_linear":77805.0,"score_lmetric":82489},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":4684,"score_linear":82014.0,"score_lmetric":86698},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":787,"cache_hit":0,"new_prefill":4684,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":16914,"ongoing_decode_tokens":0,"pending_prefill_tokens":7698,"num_requests":1,"active_p_offloads":0,"cached_blocks":539,"cache_hit":0,"new_prefill":4684,"score_linear":16914.0,"score_lmetric":12382},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":233348,"ongoing_decode_tokens":233348,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":330,"cache_hit":0,"new_prefill":4684,"score_linear":233348.0,"score_lmetric":9368},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":130486,"ongoing_decode_tokens":130486,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":676,"cache_hit":0,"new_prefill":4684,"score_linear":130486.0,"score_lmetric":9368},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":551,"cache_hit":0,"new_prefill":4684,"score_linear":44920.0,"score_lmetric":4684}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281739.832553998,"t_first_token_unix":1779712604.8464975,"t_done":281740.278400739,"t_done_unix":1779712605.2923436},{"request_id":"1262354:13:1293000:221","session_id":"1262354","input_length":35210,"t_proxy_recv":281740.382075581,"t_decision_unix":1779712605.396017,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9888099971598977,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":394,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":65567,"ongoing_decode_tokens":0,"pending_prefill_tokens":42527,"num_requests":2,"active_p_offloads":0,"cached_blocks":472,"cache_hit":0,"new_prefill":35210,"score_linear":65567.0,"score_lmetric":155474},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":35210,"score_linear":77805.0,"score_lmetric":113015},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":35210,"score_linear":82014.0,"score_lmetric":117224},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":796,"cache_hit":0,"new_prefill":35210,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":547,"cache_hit":0,"new_prefill":35210,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":233348,"ongoing_decode_tokens":233348,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":330,"cache_hit":0,"new_prefill":35210,"score_linear":233348.0,"score_lmetric":70420},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":138850,"ongoing_decode_tokens":42745,"pending_prefill_tokens":8041,"num_requests":2,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":35210,"score_linear":138850.0,"score_lmetric":86502},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":551,"cache_hit":34816,"new_prefill":394,"score_linear":10104.0,"score_lmetric":394}],"chosen_score_linear":10104.0,"chosen_score_lmetric":394,"t_first_token":281740.468423208,"t_first_token_unix":1779712605.4823663,"t_done":281741.356025608,"t_done_unix":1779712606.369969},{"request_id":"1287100:1:1287100:196","session_id":"1287100","input_length":77805,"t_proxy_recv":281720.688578322,"t_decision_unix":1779712585.7025192,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":77805,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13962,"ongoing_decode_tokens":0,"pending_prefill_tokens":13962,"num_requests":1,"active_p_offloads":0,"cached_blocks":399,"cache_hit":0,"new_prefill":77805,"score_linear":13962.0,"score_lmetric":91767},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":77805,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49619,"ongoing_decode_tokens":49619,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":77805,"score_linear":49619.0,"score_lmetric":77805},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":126737,"ongoing_decode_tokens":0,"pending_prefill_tokens":98577,"num_requests":3,"active_p_offloads":0,"cached_blocks":572,"cache_hit":0,"new_prefill":77805,"score_linear":126737.0,"score_lmetric":529146},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":497,"cache_hit":0,"new_prefill":77805,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":0,"pending_prefill_tokens":119045,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":77805,"score_linear":232709.0,"score_lmetric":393700},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":152913,"ongoing_decode_tokens":110649,"pending_prefill_tokens":10008,"num_requests":3,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":77805,"score_linear":152913.0,"score_lmetric":263439},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":31898,"ongoing_decode_tokens":31898,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":521,"cache_hit":0,"new_prefill":77805,"score_linear":31898.0,"score_lmetric":77805}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281741.410067333,"t_first_token_unix":1779712606.424011,"t_done":281741.931508682,"t_done_unix":1779712606.9454522},{"request_id":"1291100:2:1293458:223","session_id":"1291100","input_length":2572,"t_proxy_recv":281742.02665986,"t_decision_unix":1779712607.0406013,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":1536,"affinity_cache_ratio":0.5972006220839814,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":1036,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":65567,"ongoing_decode_tokens":0,"pending_prefill_tokens":42527,"num_requests":2,"active_p_offloads":0,"cached_blocks":472,"cache_hit":0,"new_prefill":2572,"score_linear":65567.0,"score_lmetric":90198},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":698,"cache_hit":0,"new_prefill":2572,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":2572,"score_linear":82014.0,"score_lmetric":84586},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":796,"cache_hit":0,"new_prefill":2572,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":547,"cache_hit":1536,"new_prefill":1036,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":233348,"ongoing_decode_tokens":233348,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":330,"cache_hit":0,"new_prefill":2572,"score_linear":233348.0,"score_lmetric":5144},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":138850,"ongoing_decode_tokens":42745,"pending_prefill_tokens":8041,"num_requests":2,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":2572,"score_linear":138850.0,"score_lmetric":21226},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":551,"cache_hit":0,"new_prefill":2572,"score_linear":44920.0,"score_lmetric":2572}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":281742.107235269,"t_first_token_unix":1779712607.1211786,"t_done":281742.258527377,"t_done_unix":1779712607.2724705},{"request_id":"1293507:1:1293507:224","session_id":"1293507","input_length":6460,"t_proxy_recv":281742.213511251,"t_decision_unix":1779712607.2274523,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":6460,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":65567,"ongoing_decode_tokens":0,"pending_prefill_tokens":42527,"num_requests":2,"active_p_offloads":0,"cached_blocks":472,"cache_hit":0,"new_prefill":6460,"score_linear":65567.0,"score_lmetric":97974},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":698,"cache_hit":0,"new_prefill":6460,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":6460,"score_linear":82014.0,"score_lmetric":88474},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":796,"cache_hit":0,"new_prefill":6460,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":2572,"ongoing_decode_tokens":2572,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":547,"cache_hit":0,"new_prefill":6460,"score_linear":2572.0,"score_lmetric":6460},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":233348,"ongoing_decode_tokens":233348,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":330,"cache_hit":0,"new_prefill":6460,"score_linear":233348.0,"score_lmetric":12920},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":138850,"ongoing_decode_tokens":42745,"pending_prefill_tokens":8041,"num_requests":2,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":6460,"score_linear":138850.0,"score_lmetric":29002},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":551,"cache_hit":0,"new_prefill":6460,"score_linear":44920.0,"score_lmetric":6460}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281742.644753015,"t_first_token_unix":1779712607.6586964,"t_done":281742.765821696,"t_done_unix":1779712607.7797647},{"request_id":"1258859:4:1282900:176","session_id":"1258859","input_length":118736,"t_proxy_recv":281736.193886729,"t_decision_unix":1779712601.2078283,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":118272,"affinity_cache_ratio":0.9960921708664601,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":118272,"estimated_new_tokens":464,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":472,"cache_hit":0,"new_prefill":118736,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":118736,"score_linear":77805.0,"score_lmetric":196541},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":118736,"score_linear":82014.0,"score_lmetric":200750},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15547,"ongoing_decode_tokens":15547,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":786,"cache_hit":0,"new_prefill":118736,"score_linear":15547.0,"score_lmetric":118736},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":524,"cache_hit":0,"new_prefill":118736,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":114612,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":330,"cache_hit":118272,"new_prefill":464,"score_linear":-3660.0,"score_lmetric":464},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":130486,"ongoing_decode_tokens":87741,"pending_prefill_tokens":3833,"num_requests":2,"active_p_offloads":0,"cached_blocks":676,"cache_hit":0,"new_prefill":118736,"score_linear":130486.0,"score_lmetric":245138},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":112378,"ongoing_decode_tokens":112378,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":551,"cache_hit":0,"new_prefill":118736,"score_linear":112378.0,"score_lmetric":356208}],"chosen_score_linear":-3660.0,"chosen_score_lmetric":464,"t_first_token":281736.435488351,"t_first_token_unix":1779712601.4494326,"t_done":281742.971418952,"t_done_unix":1779712607.9853625},{"request_id":"1290365:2:1292000:217","session_id":"1290365","input_length":23414,"t_proxy_recv":281737.23337687,"t_decision_unix":1779712602.247318,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9840266507217904,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":23040,"estimated_new_tokens":374,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":42153,"ongoing_decode_tokens":0,"pending_prefill_tokens":42153,"num_requests":1,"active_p_offloads":0,"cached_blocks":472,"cache_hit":23040,"new_prefill":374,"score_linear":19113.0,"score_lmetric":42527},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":23414,"score_linear":77805.0,"score_lmetric":101219},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":23414,"score_linear":82014.0,"score_lmetric":105428},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15969,"ongoing_decode_tokens":15969,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":786,"cache_hit":0,"new_prefill":23414,"score_linear":15969.0,"score_lmetric":23414},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":8144,"ongoing_decode_tokens":0,"pending_prefill_tokens":8144,"num_requests":1,"active_p_offloads":0,"cached_blocks":524,"cache_hit":0,"new_prefill":23414,"score_linear":8144.0,"score_lmetric":31558},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":233348,"ongoing_decode_tokens":233348,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":330,"cache_hit":0,"new_prefill":23414,"score_linear":233348.0,"score_lmetric":46828},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":130486,"ongoing_decode_tokens":130486,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":676,"cache_hit":0,"new_prefill":23414,"score_linear":130486.0,"score_lmetric":46828},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":77279,"ongoing_decode_tokens":77279,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":551,"cache_hit":0,"new_prefill":23414,"score_linear":77279.0,"score_lmetric":46828}],"chosen_score_linear":19113.0,"chosen_score_lmetric":42527,"t_first_token":281743.650163359,"t_first_token_unix":1779712608.6641061,"t_done":281744.083264565,"t_done_unix":1779712609.0972083},{"request_id":"1291944:2:1293798:227","session_id":"1291944","input_length":12388,"t_proxy_recv":281743.298536549,"t_decision_unix":1779712608.3124776,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":7680,"affinity_cache_ratio":0.6199547949628673,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":4708,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":65567,"ongoing_decode_tokens":0,"pending_prefill_tokens":42527,"num_requests":2,"active_p_offloads":0,"cached_blocks":472,"cache_hit":0,"new_prefill":12388,"score_linear":65567.0,"score_lmetric":109830},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":698,"cache_hit":0,"new_prefill":12388,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":12388,"score_linear":82014.0,"score_lmetric":94402},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":28796,"ongoing_decode_tokens":0,"pending_prefill_tokens":12924,"num_requests":1,"active_p_offloads":0,"cached_blocks":808,"cache_hit":0,"new_prefill":12388,"score_linear":28796.0,"score_lmetric":25312},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":549,"cache_hit":7680,"new_prefill":4708,"score_linear":-7680.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":114612,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":331,"cache_hit":0,"new_prefill":12388,"score_linear":114612.0,"score_lmetric":12388},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":138850,"ongoing_decode_tokens":42745,"pending_prefill_tokens":8041,"num_requests":2,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":12388,"score_linear":138850.0,"score_lmetric":40858},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":85198,"ongoing_decode_tokens":44920,"pending_prefill_tokens":8022,"num_requests":2,"active_p_offloads":0,"cached_blocks":551,"cache_hit":0,"new_prefill":12388,"score_linear":85198.0,"score_lmetric":40820}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":281743.771107218,"t_first_token_unix":1779712608.7850504,"t_done":281744.112639996,"t_done_unix":1779712609.1265829},{"request_id":"1260327:6:1291827:215","session_id":"1260327","input_length":42153,"t_proxy_recv":281736.666309994,"t_decision_unix":1779712601.6802514,"policy":"unified","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":0,"affinity_cache_hit":41984,"affinity_cache_ratio":0.9959907954356748,"affinity_num_requests":3,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":42153,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":472,"cache_hit":0,"new_prefill":42153,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":42153,"score_linear":77805.0,"score_lmetric":119958},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":42153,"score_linear":82014.0,"score_lmetric":124167},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15969,"ongoing_decode_tokens":15969,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":786,"cache_hit":0,"new_prefill":42153,"score_linear":15969.0,"score_lmetric":42153},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":524,"cache_hit":0,"new_prefill":42153,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":233348,"ongoing_decode_tokens":233348,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":330,"cache_hit":0,"new_prefill":42153,"score_linear":233348.0,"score_lmetric":84306},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":130486,"ongoing_decode_tokens":87741,"pending_prefill_tokens":3833,"num_requests":2,"active_p_offloads":0,"cached_blocks":676,"cache_hit":0,"new_prefill":42153,"score_linear":130486.0,"score_lmetric":91972},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":112378,"ongoing_decode_tokens":112378,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":551,"cache_hit":41984,"new_prefill":169,"score_linear":70394.0,"score_lmetric":507}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281743.646521887,"t_first_token_unix":1779712608.6604655,"t_done":281744.427293568,"t_done_unix":1779712609.4412367},{"request_id":"1266668:8:1292408:218","session_id":"1266668","input_length":96105,"t_proxy_recv":281740.242419082,"t_decision_unix":1779712605.2563605,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":88064,"affinity_cache_ratio":0.916331096196868,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":88064,"estimated_new_tokens":8041,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":65567,"ongoing_decode_tokens":0,"pending_prefill_tokens":42527,"num_requests":2,"active_p_offloads":0,"cached_blocks":472,"cache_hit":4096,"new_prefill":92009,"score_linear":61471.0,"score_lmetric":269072},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":96105,"score_linear":77805.0,"score_lmetric":173910},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":96105,"score_linear":82014.0,"score_lmetric":178119},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":4684,"ongoing_decode_tokens":4684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":787,"cache_hit":7680,"new_prefill":88425,"score_linear":-2996.0,"score_lmetric":88425},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":547,"cache_hit":0,"new_prefill":96105,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":233348,"ongoing_decode_tokens":233348,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":330,"cache_hit":0,"new_prefill":96105,"score_linear":233348.0,"score_lmetric":192210},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":42745,"ongoing_decode_tokens":42745,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":691,"cache_hit":88064,"new_prefill":8041,"score_linear":-45319.0,"score_lmetric":8041},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":551,"cache_hit":7680,"new_prefill":88425,"score_linear":37240.0,"score_lmetric":88425}],"chosen_score_linear":-45319.0,"chosen_score_lmetric":8041,"t_first_token":281744.817400796,"t_first_token_unix":1779712609.8313446,"t_done":281745.989743363,"t_done_unix":1779712611.0036867},{"request_id":"1290365:3:1293216:222","session_id":"1290365","input_length":24482,"t_proxy_recv":281744.088799257,"t_decision_unix":1779712609.1027405,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":23040,"affinity_cache_ratio":0.941099583367372,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":23040,"estimated_new_tokens":1442,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":42153,"ongoing_decode_tokens":42153,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":472,"cache_hit":23040,"new_prefill":1442,"score_linear":19113.0,"score_lmetric":1442},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":698,"cache_hit":0,"new_prefill":24482,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":24482,"score_linear":82014.0,"score_lmetric":106496},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":28796,"ongoing_decode_tokens":0,"pending_prefill_tokens":12924,"num_requests":1,"active_p_offloads":0,"cached_blocks":808,"cache_hit":0,"new_prefill":24482,"score_linear":28796.0,"score_lmetric":37406},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":12388,"ongoing_decode_tokens":12388,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":549,"cache_hit":0,"new_prefill":24482,"score_linear":12388.0,"score_lmetric":24482},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":114612,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":331,"cache_hit":0,"new_prefill":24482,"score_linear":114612.0,"score_lmetric":24482},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":138850,"ongoing_decode_tokens":42745,"pending_prefill_tokens":8041,"num_requests":2,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":24482,"score_linear":138850.0,"score_lmetric":65046},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":85198,"ongoing_decode_tokens":44920,"pending_prefill_tokens":8022,"num_requests":2,"active_p_offloads":0,"cached_blocks":551,"cache_hit":0,"new_prefill":24482,"score_linear":85198.0,"score_lmetric":65008}],"chosen_score_linear":19113.0,"chosen_score_lmetric":1442,"t_first_token":281744.355090068,"t_first_token_unix":1779712609.3690336,"t_done":281746.453291951,"t_done_unix":1779712611.467235},{"request_id":"1272313:8:1293738:226","session_id":"1272313","input_length":28796,"t_proxy_recv":281743.150900592,"t_decision_unix":1779712608.164842,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":15872,"affinity_cache_ratio":0.5511876649534657,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":15872,"estimated_new_tokens":12924,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":65567,"ongoing_decode_tokens":0,"pending_prefill_tokens":42527,"num_requests":2,"active_p_offloads":0,"cached_blocks":472,"cache_hit":4096,"new_prefill":24700,"score_linear":61471.0,"score_lmetric":134454},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":698,"cache_hit":0,"new_prefill":28796,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":28796,"score_linear":82014.0,"score_lmetric":110810},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":808,"cache_hit":15872,"new_prefill":12924,"score_linear":-15872.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":549,"cache_hit":0,"new_prefill":28796,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":114612,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":331,"cache_hit":0,"new_prefill":28796,"score_linear":114612.0,"score_lmetric":28796},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":138850,"ongoing_decode_tokens":42745,"pending_prefill_tokens":8041,"num_requests":2,"active_p_offloads":0,"cached_blocks":691,"cache_hit":10752,"new_prefill":18044,"score_linear":128098.0,"score_lmetric":52170},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":85198,"ongoing_decode_tokens":44920,"pending_prefill_tokens":8022,"num_requests":2,"active_p_offloads":0,"cached_blocks":551,"cache_hit":7680,"new_prefill":21116,"score_linear":77518.0,"score_lmetric":58276}],"chosen_score_linear":-15872.0,"chosen_score_lmetric":0,"t_first_token":281745.33366169,"t_first_token_unix":1779712610.347605,"t_done":281747.150994336,"t_done_unix":1779712612.1649377},{"request_id":"1269373:7:1290314:207","session_id":"1269373","input_length":42745,"t_proxy_recv":281736.010549979,"t_decision_unix":1779712601.0244915,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":38912,"affinity_cache_ratio":0.9103286934144345,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":38912,"estimated_new_tokens":3833,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":472,"cache_hit":4096,"new_prefill":38649,"score_linear":-4096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":42745,"score_linear":77805.0,"score_lmetric":120550},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":42745,"score_linear":82014.0,"score_lmetric":124759},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15547,"ongoing_decode_tokens":15547,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":786,"cache_hit":7680,"new_prefill":35065,"score_linear":7867.0,"score_lmetric":35065},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":524,"cache_hit":0,"new_prefill":42745,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":42745,"score_linear":232709.0,"score_lmetric":85490},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":87741,"ongoing_decode_tokens":87741,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":676,"cache_hit":38912,"new_prefill":3833,"score_linear":48829.0,"score_lmetric":3833},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":112378,"ongoing_decode_tokens":80019,"pending_prefill_tokens":103,"num_requests":3,"active_p_offloads":0,"cached_blocks":551,"cache_hit":7680,"new_prefill":35065,"score_linear":104698.0,"score_lmetric":105504}],"chosen_score_linear":48829.0,"chosen_score_lmetric":3833,"t_first_token":281737.081334046,"t_first_token_unix":1779712602.0952778,"t_done":281747.457276253,"t_done_unix":1779712612.4712198},{"request_id":"1262354:14:1294243:228","session_id":"1262354","input_length":35435,"t_proxy_recv":281744.467324751,"t_decision_unix":1779712609.481266,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9825313955129109,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":619,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":24482,"ongoing_decode_tokens":24482,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":554,"cache_hit":0,"new_prefill":35435,"score_linear":24482.0,"score_lmetric":35435},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":698,"cache_hit":0,"new_prefill":35435,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":35435,"score_linear":82014.0,"score_lmetric":117449},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":28796,"ongoing_decode_tokens":0,"pending_prefill_tokens":12924,"num_requests":1,"active_p_offloads":0,"cached_blocks":808,"cache_hit":0,"new_prefill":35435,"score_linear":28796.0,"score_lmetric":48359},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":558,"cache_hit":0,"new_prefill":35435,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":114612,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":331,"cache_hit":0,"new_prefill":35435,"score_linear":114612.0,"score_lmetric":35435},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":138850,"ongoing_decode_tokens":42745,"pending_prefill_tokens":8041,"num_requests":2,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":35435,"score_linear":138850.0,"score_lmetric":86952},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":85198,"ongoing_decode_tokens":44920,"pending_prefill_tokens":8022,"num_requests":2,"active_p_offloads":0,"cached_blocks":551,"cache_hit":34816,"new_prefill":619,"score_linear":50382.0,"score_lmetric":17282}],"chosen_score_linear":50382.0,"chosen_score_lmetric":17282,"t_first_token":281744.903568302,"t_first_token_unix":1779712609.9175115,"t_done":281747.477953053,"t_done_unix":1779712612.4918962},{"request_id":"1294611:1:1294611:229","session_id":"1294611","input_length":21947,"t_proxy_recv":281745.765047525,"t_decision_unix":1779712610.7789888,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":21947,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":24482,"ongoing_decode_tokens":24482,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":554,"cache_hit":0,"new_prefill":21947,"score_linear":24482.0,"score_lmetric":21947},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":698,"cache_hit":0,"new_prefill":21947,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":21947,"score_linear":82014.0,"score_lmetric":103961},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":28796,"ongoing_decode_tokens":28796,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":808,"cache_hit":12800,"new_prefill":9147,"score_linear":15996.0,"score_lmetric":9147},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":558,"cache_hit":0,"new_prefill":21947,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":114612,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":331,"cache_hit":0,"new_prefill":21947,"score_linear":114612.0,"score_lmetric":21947},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":138850,"ongoing_decode_tokens":138850,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":21947,"score_linear":138850.0,"score_lmetric":43894},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":120633,"ongoing_decode_tokens":120633,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":551,"cache_hit":0,"new_prefill":21947,"score_linear":120633.0,"score_lmetric":65841}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281748.138836793,"t_first_token_unix":1779712613.15278,"t_done":281748.809781321,"t_done_unix":1779712613.8237245},{"request_id":"1279412:6:1293655:225","session_id":"1279412","input_length":40278,"t_proxy_recv":281742.793178523,"t_decision_unix":1779712607.80712,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":32256,"affinity_cache_ratio":0.8008342022940563,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":32256,"estimated_new_tokens":8022,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":65567,"ongoing_decode_tokens":0,"pending_prefill_tokens":42527,"num_requests":2,"active_p_offloads":0,"cached_blocks":472,"cache_hit":0,"new_prefill":40278,"score_linear":65567.0,"score_lmetric":165610},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":698,"cache_hit":0,"new_prefill":40278,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":40278,"score_linear":82014.0,"score_lmetric":122292},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":808,"cache_hit":0,"new_prefill":40278,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":549,"cache_hit":0,"new_prefill":40278,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":233348,"ongoing_decode_tokens":233348,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":330,"cache_hit":0,"new_prefill":40278,"score_linear":233348.0,"score_lmetric":80556},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":138850,"ongoing_decode_tokens":42745,"pending_prefill_tokens":8041,"num_requests":2,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":40278,"score_linear":138850.0,"score_lmetric":96638},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":551,"cache_hit":32256,"new_prefill":8022,"score_linear":12664.0,"score_lmetric":8022}],"chosen_score_linear":12664.0,"chosen_score_lmetric":8022,"t_first_token":281744.811565502,"t_first_token_unix":1779712609.8255088,"t_done":281750.416990447,"t_done_unix":1779712615.4309337},{"request_id":"1295690:1:1295690:232","session_id":"1295690","input_length":10857,"t_proxy_recv":281749.663025748,"t_decision_unix":1779712614.6769671,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":10857,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":557,"cache_hit":0,"new_prefill":10857,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":741,"cache_hit":0,"new_prefill":10857,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":10857,"score_linear":82014.0,"score_lmetric":92871},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":10857,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":558,"cache_hit":0,"new_prefill":10857,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":114612,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":331,"cache_hit":0,"new_prefill":10857,"score_linear":114612.0,"score_lmetric":10857},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":147397,"ongoing_decode_tokens":0,"pending_prefill_tokens":8645,"num_requests":2,"active_p_offloads":0,"cached_blocks":714,"cache_hit":0,"new_prefill":10857,"score_linear":147397.0,"score_lmetric":39004},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":85198,"ongoing_decode_tokens":85198,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":552,"cache_hit":0,"new_prefill":10857,"score_linear":85198.0,"score_lmetric":21714}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281750.546319714,"t_first_token_unix":1779712615.5602632,"t_done":281750.987211364,"t_done_unix":1779712616.0011544},{"request_id":"1266668:9:1295576:231","session_id":"1266668","input_length":96717,"t_proxy_recv":281749.330695127,"t_decision_unix":1779712614.3446364,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":95744,"affinity_cache_ratio":0.9899397210418024,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":95744,"estimated_new_tokens":973,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":557,"cache_hit":4096,"new_prefill":92621,"score_linear":-4096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":741,"cache_hit":0,"new_prefill":96717,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":96717,"score_linear":82014.0,"score_lmetric":178731},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":7680,"new_prefill":89037,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":558,"cache_hit":0,"new_prefill":96717,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":114612,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":331,"cache_hit":0,"new_prefill":96717,"score_linear":114612.0,"score_lmetric":96717},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50680,"ongoing_decode_tokens":0,"pending_prefill_tokens":7672,"num_requests":1,"active_p_offloads":0,"cached_blocks":714,"cache_hit":95744,"new_prefill":973,"score_linear":-45064.0,"score_lmetric":8645},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":85198,"ongoing_decode_tokens":85198,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":552,"cache_hit":7680,"new_prefill":89037,"score_linear":77518.0,"score_lmetric":178074}],"chosen_score_linear":-45064.0,"chosen_score_lmetric":8645,"t_first_token":281751.051678648,"t_first_token_unix":1779712616.065622,"t_done":281751.473863888,"t_done_unix":1779712616.4878073},{"request_id":"1294611:2:1295852:233","session_id":"1294611","input_length":23610,"t_proxy_recv":281750.171640746,"t_decision_unix":1779712615.1855822,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":22016,"affinity_cache_ratio":0.9324862346463363,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":22016,"estimated_new_tokens":1594,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":557,"cache_hit":0,"new_prefill":23610,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":741,"cache_hit":22016,"new_prefill":1594,"score_linear":-22016.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":23610,"score_linear":82014.0,"score_lmetric":105624},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":12800,"new_prefill":10810,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":10857,"ongoing_decode_tokens":0,"pending_prefill_tokens":10857,"num_requests":1,"active_p_offloads":0,"cached_blocks":558,"cache_hit":0,"new_prefill":23610,"score_linear":10857.0,"score_lmetric":34467},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":114612,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":331,"cache_hit":0,"new_prefill":23610,"score_linear":114612.0,"score_lmetric":23610},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":147397,"ongoing_decode_tokens":0,"pending_prefill_tokens":8645,"num_requests":2,"active_p_offloads":0,"cached_blocks":714,"cache_hit":0,"new_prefill":23610,"score_linear":147397.0,"score_lmetric":64510},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":85198,"ongoing_decode_tokens":85198,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":552,"cache_hit":0,"new_prefill":23610,"score_linear":85198.0,"score_lmetric":47220}],"chosen_score_linear":-22016.0,"chosen_score_lmetric":0,"t_first_token":281750.55885444,"t_first_token_unix":1779712615.5727978,"t_done":281751.933015684,"t_done_unix":1779712616.9469588},{"request_id":"1289570:1:1289570:204","session_id":"1289570","input_length":82014,"t_proxy_recv":281729.004985637,"t_decision_unix":1779712594.0189266,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":82014,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":427,"cache_hit":0,"new_prefill":82014,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":82014,"score_linear":77805.0,"score_lmetric":159819},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":82014,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":104111,"ongoing_decode_tokens":104111,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":767,"cache_hit":0,"new_prefill":82014,"score_linear":104111.0,"score_lmetric":164028},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":9610,"ongoing_decode_tokens":0,"pending_prefill_tokens":8074,"num_requests":1,"active_p_offloads":0,"cached_blocks":497,"cache_hit":0,"new_prefill":82014,"score_linear":9610.0,"score_lmetric":90088},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":82014,"score_linear":232709.0,"score_lmetric":164028},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":118843,"ongoing_decode_tokens":118843,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":617,"cache_hit":0,"new_prefill":82014,"score_linear":118843.0,"score_lmetric":164028},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":77111,"ongoing_decode_tokens":0,"pending_prefill_tokens":14135,"num_requests":2,"active_p_offloads":0,"cached_blocks":550,"cache_hit":0,"new_prefill":82014,"score_linear":77111.0,"score_lmetric":192298}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281751.70704248,"t_first_token_unix":1779712616.720987,"t_done":281752.227529254,"t_done_unix":1779712617.2414727},{"request_id":"1275274:1:1275274:136","session_id":"1275274","input_length":114612,"t_proxy_recv":281679.551590323,"t_decision_unix":1779712544.5655315,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":114612,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":114612,"score_linear":135235.0,"score_lmetric":455662},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":114612,"score_linear":120167.0,"score_lmetric":441910},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":114612,"score_linear":150574.0,"score_lmetric":464836},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":475,"cache_hit":0,"new_prefill":114612,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":114612,"score_linear":121069.0,"score_lmetric":416084},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":114612,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":445,"cache_hit":0,"new_prefill":114612,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":370,"cache_hit":0,"new_prefill":114612,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281721.898465392,"t_first_token_unix":1779712586.9124093,"t_done":281753.601332026,"t_done_unix":1779712618.615276},{"request_id":"1296872:1:1296872:235","session_id":"1296872","input_length":624,"t_proxy_recv":281753.726528718,"t_decision_unix":1779712618.74047,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":624,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":557,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":744,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":855,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":556,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50680,"ongoing_decode_tokens":50680,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":715,"cache_hit":0,"new_prefill":624,"score_linear":50680.0,"score_lmetric":624},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80589,"ongoing_decode_tokens":80589,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":568,"cache_hit":0,"new_prefill":624,"score_linear":80589.0,"score_lmetric":1248}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281753.79398014,"t_first_token_unix":1779712618.8079233,"t_done":281753.897976011,"t_done_unix":1779712618.9119196},{"request_id":"1296983:1:1296983:236","session_id":"1296983","input_length":1791,"t_proxy_recv":281754.159859239,"t_decision_unix":1779712619.1738007,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1791,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":557,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":744,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":556,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50680,"ongoing_decode_tokens":50680,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":715,"cache_hit":0,"new_prefill":1791,"score_linear":50680.0,"score_lmetric":1791},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80589,"ongoing_decode_tokens":80589,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":568,"cache_hit":0,"new_prefill":1791,"score_linear":80589.0,"score_lmetric":3582}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281754.280453454,"t_first_token_unix":1779712619.2943969,"t_done":281754.505563074,"t_done_unix":1779712619.5195065},{"request_id":"1262354:15:1296106:234","session_id":"1262354","input_length":35669,"t_proxy_recv":281751.114635081,"t_decision_unix":1779712616.1285765,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9904398777650061,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35328,"estimated_new_tokens":341,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":557,"cache_hit":0,"new_prefill":35669,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23610,"ongoing_decode_tokens":23610,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":741,"cache_hit":0,"new_prefill":35669,"score_linear":23610.0,"score_lmetric":35669},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":35669,"score_linear":82014.0,"score_lmetric":117683},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":35669,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":35669,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":114612,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":331,"cache_hit":0,"new_prefill":35669,"score_linear":114612.0,"score_lmetric":35669},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":147397,"ongoing_decode_tokens":147397,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":714,"cache_hit":0,"new_prefill":35669,"score_linear":147397.0,"score_lmetric":71338},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":568,"cache_hit":35328,"new_prefill":341,"score_linear":9592.0,"score_lmetric":341}],"chosen_score_linear":9592.0,"chosen_score_lmetric":341,"t_first_token":281751.201410045,"t_first_token_unix":1779712616.2153535,"t_done":281754.807854425,"t_done_unix":1779712619.8217976},{"request_id":"1269373:8:1295332:230","session_id":"1269373","input_length":50680,"t_proxy_recv":281748.310924907,"t_decision_unix":1779712613.324866,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":43008,"affinity_cache_ratio":0.8486187845303867,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":43008,"estimated_new_tokens":7672,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":557,"cache_hit":4096,"new_prefill":46584,"score_linear":-4096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":21947,"ongoing_decode_tokens":21947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":698,"cache_hit":0,"new_prefill":50680,"score_linear":21947.0,"score_lmetric":50680},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":50680,"score_linear":82014.0,"score_lmetric":132694},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":7680,"new_prefill":43000,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":558,"cache_hit":0,"new_prefill":50680,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114612,"ongoing_decode_tokens":114612,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":331,"cache_hit":0,"new_prefill":50680,"score_linear":114612.0,"score_lmetric":50680},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":714,"cache_hit":43008,"new_prefill":7672,"score_linear":-43008.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":85198,"ongoing_decode_tokens":85198,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":552,"cache_hit":7680,"new_prefill":43000,"score_linear":77518.0,"score_lmetric":86000}],"chosen_score_linear":-43008.0,"chosen_score_lmetric":0,"t_first_token":281750.645969318,"t_first_token_unix":1779712615.6599123,"t_done":281755.212031863,"t_done_unix":1779712620.225975},{"request_id":"1260327:7:1297673:242","session_id":"1260327","input_length":42363,"t_proxy_recv":281756.347967068,"t_decision_unix":1779712621.3619087,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":41984,"affinity_cache_ratio":0.99105351367939,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":41984,"estimated_new_tokens":379,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":557,"cache_hit":41984,"new_prefill":379,"score_linear":-41984.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23863,"ongoing_decode_tokens":23863,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":744,"cache_hit":0,"new_prefill":42363,"score_linear":23863.0,"score_lmetric":42363},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":42363,"score_linear":62414.0,"score_lmetric":85833},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":836,"cache_hit":0,"new_prefill":42363,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":59507,"ongoing_decode_tokens":0,"pending_prefill_tokens":59507,"num_requests":1,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":42363,"score_linear":59507.0,"score_lmetric":101870},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":556,"cache_hit":0,"new_prefill":42363,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":97145,"ongoing_decode_tokens":0,"pending_prefill_tokens":889,"num_requests":1,"active_p_offloads":0,"cached_blocks":730,"cache_hit":0,"new_prefill":42363,"score_linear":97145.0,"score_lmetric":43252},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105113,"ongoing_decode_tokens":44920,"pending_prefill_tokens":19745,"num_requests":2,"active_p_offloads":0,"cached_blocks":569,"cache_hit":41984,"new_prefill":379,"score_linear":63129.0,"score_lmetric":40248}],"chosen_score_linear":-41984.0,"chosen_score_lmetric":0,"t_first_token":281756.503398758,"t_first_token_unix":1779712621.5173419,"t_done":281756.732353022,"t_done_unix":1779712621.7462962},{"request_id":"1290442:2:1298058:246","session_id":"1290442","input_length":11994,"t_proxy_recv":281757.696724961,"t_decision_unix":1779712622.7106664,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":7168,"affinity_cache_ratio":0.5976321494080373,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":7168,"estimated_new_tokens":4826,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13992,"ongoing_decode_tokens":0,"pending_prefill_tokens":12456,"num_requests":1,"active_p_offloads":0,"cached_blocks":557,"cache_hit":0,"new_prefill":11994,"score_linear":13992.0,"score_lmetric":24450},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23863,"ongoing_decode_tokens":23863,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":744,"cache_hit":0,"new_prefill":11994,"score_linear":23863.0,"score_lmetric":11994},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":11994,"score_linear":62414.0,"score_lmetric":55464},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29149,"ongoing_decode_tokens":29149,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":836,"cache_hit":7168,"new_prefill":4826,"score_linear":21981.0,"score_lmetric":4826},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":0,"pending_prefill_tokens":68337,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":11994,"score_linear":81137.0,"score_lmetric":160662},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":556,"cache_hit":0,"new_prefill":11994,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":97145,"ongoing_decode_tokens":97145,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":730,"cache_hit":0,"new_prefill":11994,"score_linear":97145.0,"score_lmetric":11994},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105113,"ongoing_decode_tokens":44920,"pending_prefill_tokens":19745,"num_requests":2,"active_p_offloads":0,"cached_blocks":569,"cache_hit":0,"new_prefill":11994,"score_linear":105113.0,"score_lmetric":63478}],"chosen_score_linear":21981.0,"chosen_score_lmetric":4826,"t_first_token":281758.211156117,"t_first_token_unix":1779712623.2250996,"t_done":281758.333132411,"t_done_unix":1779712623.3470762},{"request_id":"1272313:9:1297810:243","session_id":"1272313","input_length":29149,"t_proxy_recv":281756.842770785,"t_decision_unix":1779712621.856712,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":28672,"affinity_cache_ratio":0.9836358022573674,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":28672,"estimated_new_tokens":477,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":557,"cache_hit":4096,"new_prefill":25053,"score_linear":-4096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23863,"ongoing_decode_tokens":23863,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":744,"cache_hit":0,"new_prefill":29149,"score_linear":23863.0,"score_lmetric":29149},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":29149,"score_linear":62414.0,"score_lmetric":72619},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":836,"cache_hit":28672,"new_prefill":477,"score_linear":-28672.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":59507,"ongoing_decode_tokens":0,"pending_prefill_tokens":59507,"num_requests":1,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":29149,"score_linear":59507.0,"score_lmetric":88656},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":556,"cache_hit":0,"new_prefill":29149,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":97145,"ongoing_decode_tokens":97145,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":730,"cache_hit":10752,"new_prefill":18397,"score_linear":86393.0,"score_lmetric":18397},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105113,"ongoing_decode_tokens":44920,"pending_prefill_tokens":19745,"num_requests":2,"active_p_offloads":0,"cached_blocks":569,"cache_hit":7680,"new_prefill":21469,"score_linear":97433.0,"score_lmetric":82428}],"chosen_score_linear":-28672.0,"chosen_score_lmetric":0,"t_first_token":281756.929065729,"t_first_token_unix":1779712621.9430091,"t_done":281758.353430927,"t_done_unix":1779712623.3673742},{"request_id":"1294611:3:1297534:240","session_id":"1294611","input_length":23863,"t_proxy_recv":281756.004577199,"t_decision_unix":1779712621.0185184,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":23552,"affinity_cache_ratio":0.9869672715081926,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":23552,"estimated_new_tokens":311,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":557,"cache_hit":0,"new_prefill":23863,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":744,"cache_hit":23552,"new_prefill":311,"score_linear":-23552.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":23863,"score_linear":62414.0,"score_lmetric":67333},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":836,"cache_hit":12800,"new_prefill":11063,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":59507,"ongoing_decode_tokens":0,"pending_prefill_tokens":59507,"num_requests":1,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":23863,"score_linear":59507.0,"score_lmetric":83370},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":556,"cache_hit":0,"new_prefill":23863,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":730,"cache_hit":0,"new_prefill":23863,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105113,"ongoing_decode_tokens":44920,"pending_prefill_tokens":19745,"num_requests":2,"active_p_offloads":0,"cached_blocks":569,"cache_hit":0,"new_prefill":23863,"score_linear":105113.0,"score_lmetric":87216}],"chosen_score_linear":-23552.0,"chosen_score_lmetric":0,"t_first_token":281756.059693341,"t_first_token_unix":1779712621.0736365,"t_done":281759.142860407,"t_done_unix":1779712624.1568036},{"request_id":"1297811:1:1297811:244","session_id":"1297811","input_length":13992,"t_proxy_recv":281756.84677439,"t_decision_unix":1779712621.8607163,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":12456,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":557,"cache_hit":1536,"new_prefill":12456,"score_linear":-1536.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23863,"ongoing_decode_tokens":23863,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":744,"cache_hit":0,"new_prefill":13992,"score_linear":23863.0,"score_lmetric":13992},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":13992,"score_linear":62414.0,"score_lmetric":57462},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29149,"ongoing_decode_tokens":0,"pending_prefill_tokens":477,"num_requests":1,"active_p_offloads":0,"cached_blocks":836,"cache_hit":0,"new_prefill":13992,"score_linear":29149.0,"score_lmetric":14469},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":59507,"ongoing_decode_tokens":0,"pending_prefill_tokens":59507,"num_requests":1,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":13992,"score_linear":59507.0,"score_lmetric":73499},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":556,"cache_hit":0,"new_prefill":13992,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":97145,"ongoing_decode_tokens":97145,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":730,"cache_hit":0,"new_prefill":13992,"score_linear":97145.0,"score_lmetric":13992},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105113,"ongoing_decode_tokens":44920,"pending_prefill_tokens":19745,"num_requests":2,"active_p_offloads":0,"cached_blocks":569,"cache_hit":0,"new_prefill":13992,"score_linear":105113.0,"score_lmetric":67474}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":281757.979996092,"t_first_token_unix":1779712622.9939394,"t_done":281759.281061967,"t_done_unix":1779712624.295005},{"request_id":"1266668:10:1297641:241","session_id":"1266668","input_length":97145,"t_proxy_recv":281756.265904143,"t_decision_unix":1779712621.2798457,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":96256,"affinity_cache_ratio":0.9908487312779866,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":96256,"estimated_new_tokens":889,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":557,"cache_hit":4096,"new_prefill":93049,"score_linear":-4096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23863,"ongoing_decode_tokens":23863,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":744,"cache_hit":0,"new_prefill":97145,"score_linear":23863.0,"score_lmetric":97145},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":97145,"score_linear":62414.0,"score_lmetric":140615},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":836,"cache_hit":7680,"new_prefill":89465,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":59507,"ongoing_decode_tokens":0,"pending_prefill_tokens":59507,"num_requests":1,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":97145,"score_linear":59507.0,"score_lmetric":156652},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":556,"cache_hit":0,"new_prefill":97145,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":730,"cache_hit":96256,"new_prefill":889,"score_linear":-96256.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105113,"ongoing_decode_tokens":44920,"pending_prefill_tokens":19745,"num_requests":2,"active_p_offloads":0,"cached_blocks":569,"cache_hit":7680,"new_prefill":89465,"score_linear":97433.0,"score_lmetric":218420}],"chosen_score_linear":-96256.0,"chosen_score_lmetric":0,"t_first_token":281756.749218256,"t_first_token_unix":1779712621.7631617,"t_done":281759.494647245,"t_done_unix":1779712624.508591},{"request_id":"1290365:4:1298540:247","session_id":"1290365","input_length":25103,"t_proxy_recv":281759.403158565,"t_decision_unix":1779712624.4171,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":24576,"affinity_cache_ratio":0.979006493247819,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24576,"estimated_new_tokens":527,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":581,"cache_hit":24576,"new_prefill":527,"score_linear":-24576.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":745,"cache_hit":0,"new_prefill":25103,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":25103,"score_linear":62414.0,"score_lmetric":68573},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":846,"cache_hit":0,"new_prefill":25103,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":0,"pending_prefill_tokens":68337,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":25103,"score_linear":81137.0,"score_lmetric":186880},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":556,"cache_hit":0,"new_prefill":25103,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":97145,"ongoing_decode_tokens":97145,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":730,"cache_hit":0,"new_prefill":25103,"score_linear":97145.0,"score_lmetric":25103},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105113,"ongoing_decode_tokens":44920,"pending_prefill_tokens":19745,"num_requests":2,"active_p_offloads":0,"cached_blocks":569,"cache_hit":0,"new_prefill":25103,"score_linear":105113.0,"score_lmetric":89696}],"chosen_score_linear":-24576.0,"chosen_score_lmetric":0,"t_first_token":281759.512513398,"t_first_token_unix":1779712624.5264568,"t_done":281760.584194269,"t_done_unix":1779712625.5981374},{"request_id":"1298970:1:1298970:248","session_id":"1298970","input_length":9530,"t_proxy_recv":281760.96657495,"t_decision_unix":1779712625.9805164,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9530,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":582,"cache_hit":0,"new_prefill":9530,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":745,"cache_hit":0,"new_prefill":9530,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":9530,"score_linear":62414.0,"score_lmetric":53000},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":846,"cache_hit":0,"new_prefill":9530,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":0,"pending_prefill_tokens":68337,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":9530,"score_linear":81137.0,"score_lmetric":155734},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":556,"cache_hit":0,"new_prefill":9530,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":9530,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105113,"ongoing_decode_tokens":44920,"pending_prefill_tokens":19745,"num_requests":2,"active_p_offloads":0,"cached_blocks":569,"cache_hit":0,"new_prefill":9530,"score_linear":105113.0,"score_lmetric":58550}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281761.69508258,"t_first_token_unix":1779712626.7090263,"t_done":281761.882073406,"t_done_unix":1779712626.8960166},{"request_id":"1299073:1:1299073:249","session_id":"1299073","input_length":881,"t_proxy_recv":281761.325918473,"t_decision_unix":1779712626.33986,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":881,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":582,"cache_hit":0,"new_prefill":881,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":745,"cache_hit":0,"new_prefill":881,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":881,"score_linear":62414.0,"score_lmetric":44351},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":846,"cache_hit":0,"new_prefill":881,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":0,"pending_prefill_tokens":68337,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":881,"score_linear":81137.0,"score_lmetric":138436},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":9530,"ongoing_decode_tokens":0,"pending_prefill_tokens":9530,"num_requests":1,"active_p_offloads":0,"cached_blocks":556,"cache_hit":0,"new_prefill":881,"score_linear":9530.0,"score_lmetric":10411},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":881,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105113,"ongoing_decode_tokens":44920,"pending_prefill_tokens":19745,"num_requests":2,"active_p_offloads":0,"cached_blocks":569,"cache_hit":0,"new_prefill":881,"score_linear":105113.0,"score_lmetric":41252}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281761.404154016,"t_first_token_unix":1779712626.4180973,"t_done":281762.530622952,"t_done_unix":1779712627.544566},{"request_id":"1296983:2:1299368:250","session_id":"1296983","input_length":7963,"t_proxy_recv":281762.15031223,"t_decision_unix":1779712627.1642537,"policy":"unified","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":1536,"affinity_cache_ratio":0.1928921260831345,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":6427,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":881,"ongoing_decode_tokens":881,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":582,"cache_hit":0,"new_prefill":7963,"score_linear":881.0,"score_lmetric":7963},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":745,"cache_hit":0,"new_prefill":7963,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":7963,"score_linear":62414.0,"score_lmetric":51433},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":846,"cache_hit":1536,"new_prefill":6427,"score_linear":-1536.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":0,"pending_prefill_tokens":68337,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":7963,"score_linear":81137.0,"score_lmetric":152600},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":574,"cache_hit":0,"new_prefill":7963,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":7963,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105113,"ongoing_decode_tokens":105113,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":569,"cache_hit":0,"new_prefill":7963,"score_linear":105113.0,"score_lmetric":15926}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":281762.612405036,"t_first_token_unix":1779712627.626348,"t_done":281762.806717687,"t_done_unix":1779712627.8206608},{"request_id":"1296872:2:1299394:251","session_id":"1296872","input_length":8352,"t_proxy_recv":281762.263575428,"t_decision_unix":1779712627.2775168,"policy":"unified","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":5,"affinity_cache_hit":512,"affinity_cache_ratio":0.06130268199233716,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8352,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":881,"ongoing_decode_tokens":881,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":582,"cache_hit":0,"new_prefill":8352,"score_linear":881.0,"score_lmetric":8352},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":745,"cache_hit":0,"new_prefill":8352,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":512,"new_prefill":7840,"score_linear":61902.0,"score_lmetric":51310},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":7963,"ongoing_decode_tokens":0,"pending_prefill_tokens":6427,"num_requests":1,"active_p_offloads":0,"cached_blocks":846,"cache_hit":0,"new_prefill":8352,"score_linear":7963.0,"score_lmetric":14779},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":0,"pending_prefill_tokens":68337,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":8352,"score_linear":81137.0,"score_lmetric":153378},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":574,"cache_hit":0,"new_prefill":8352,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":8352,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105113,"ongoing_decode_tokens":105113,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":569,"cache_hit":0,"new_prefill":8352,"score_linear":105113.0,"score_lmetric":16704}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281762.870522507,"t_first_token_unix":1779712627.8844657,"t_done":281763.004211951,"t_done_unix":1779712628.018155},{"request_id":"1279412:7:1297236:238","session_id":"1279412","input_length":60193,"t_proxy_recv":281755.164210652,"t_decision_unix":1779712620.1781518,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":40448,"affinity_cache_ratio":0.6719718239662419,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":19745,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":557,"cache_hit":0,"new_prefill":60193,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":744,"cache_hit":0,"new_prefill":60193,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":60193,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":836,"cache_hit":0,"new_prefill":60193,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":59507,"ongoing_decode_tokens":0,"pending_prefill_tokens":59507,"num_requests":1,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":60193,"score_linear":59507.0,"score_lmetric":119700},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":556,"cache_hit":0,"new_prefill":60193,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50680,"ongoing_decode_tokens":50680,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":715,"cache_hit":0,"new_prefill":60193,"score_linear":50680.0,"score_lmetric":60193},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":569,"cache_hit":40448,"new_prefill":19745,"score_linear":4472.0,"score_lmetric":19745}],"chosen_score_linear":4472.0,"chosen_score_lmetric":19745,"t_first_token":281761.631045172,"t_first_token_unix":1779712626.6449888,"t_done":281764.256790335,"t_done_unix":1779712629.2707343},{"request_id":"1292750:2:1299406:252","session_id":"1292750","input_length":19157,"t_proxy_recv":281762.308008615,"t_decision_unix":1779712627.3219502,"policy":"unified","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":1,"affinity_cache_hit":4608,"affinity_cache_ratio":0.24053870647804979,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":19157,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":881,"ongoing_decode_tokens":881,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":582,"cache_hit":0,"new_prefill":19157,"score_linear":881.0,"score_lmetric":19157},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":745,"cache_hit":0,"new_prefill":19157,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":19157,"score_linear":62414.0,"score_lmetric":62627},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":7963,"ongoing_decode_tokens":0,"pending_prefill_tokens":6427,"num_requests":1,"active_p_offloads":0,"cached_blocks":846,"cache_hit":4608,"new_prefill":14549,"score_linear":3355.0,"score_lmetric":20976},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":0,"pending_prefill_tokens":68337,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":19157,"score_linear":81137.0,"score_lmetric":174988},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":8352,"ongoing_decode_tokens":0,"pending_prefill_tokens":8352,"num_requests":1,"active_p_offloads":0,"cached_blocks":574,"cache_hit":0,"new_prefill":19157,"score_linear":8352.0,"score_lmetric":27509},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":19157,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105113,"ongoing_decode_tokens":105113,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":569,"cache_hit":0,"new_prefill":19157,"score_linear":105113.0,"score_lmetric":38314}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281764.249973965,"t_first_token_unix":1779712629.2639172,"t_done":281764.328196797,"t_done_unix":1779712629.3421397},{"request_id":"1272313:10:1299814:254","session_id":"1272313","input_length":29486,"t_proxy_recv":281763.683989923,"t_decision_unix":1779712628.697931,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":29184,"affinity_cache_ratio":0.9897578511836126,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":29184,"estimated_new_tokens":302,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":584,"cache_hit":4096,"new_prefill":25390,"score_linear":-4096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":19157,"ongoing_decode_tokens":0,"pending_prefill_tokens":19157,"num_requests":1,"active_p_offloads":0,"cached_blocks":745,"cache_hit":0,"new_prefill":29486,"score_linear":19157.0,"score_lmetric":48643},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":29486,"score_linear":62414.0,"score_lmetric":72956},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":858,"cache_hit":29184,"new_prefill":302,"score_linear":-29184.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":0,"pending_prefill_tokens":68337,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":29486,"score_linear":81137.0,"score_lmetric":195646},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":29486,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18893,"ongoing_decode_tokens":0,"pending_prefill_tokens":7117,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":10752,"new_prefill":18734,"score_linear":8141.0,"score_lmetric":25851},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105113,"ongoing_decode_tokens":105113,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":569,"cache_hit":7680,"new_prefill":21806,"score_linear":97433.0,"score_lmetric":43612}],"chosen_score_linear":-29184.0,"chosen_score_lmetric":0,"t_first_token":281763.799267808,"t_first_token_unix":1779712628.8132112,"t_done":281765.264513062,"t_done_unix":1779712630.2784562},{"request_id":"1277909:3:1289205:201","session_id":"1277909","input_length":44920,"t_proxy_recv":281727.704359628,"t_decision_unix":1779712592.7183008,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":31232,"affinity_cache_ratio":0.6952804986642921,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":31232,"estimated_new_tokens":13688,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":427,"cache_hit":4096,"new_prefill":40824,"score_linear":-4096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":546,"cache_hit":0,"new_prefill":44920,"score_linear":77805.0,"score_lmetric":122725},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":44920,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":104111,"ongoing_decode_tokens":89226,"pending_prefill_tokens":1061,"num_requests":2,"active_p_offloads":0,"cached_blocks":767,"cache_hit":7680,"new_prefill":37240,"score_linear":96431.0,"score_lmetric":76602},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":497,"cache_hit":0,"new_prefill":44920,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":232709,"ongoing_decode_tokens":232709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":321,"cache_hit":0,"new_prefill":44920,"score_linear":232709.0,"score_lmetric":89840},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":118843,"ongoing_decode_tokens":118843,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":617,"cache_hit":7680,"new_prefill":37240,"score_linear":111163.0,"score_lmetric":74480},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":550,"cache_hit":31232,"new_prefill":13688,"score_linear":-31232.0,"score_lmetric":0}],"chosen_score_linear":-31232.0,"chosen_score_lmetric":0,"t_first_token":281731.240058397,"t_first_token_unix":1779712596.2540019,"t_done":281765.629387412,"t_done_unix":1779712630.643331},{"request_id":"1286804:2:1300234:256","session_id":"1286804","input_length":14213,"t_proxy_recv":281765.131349188,"t_decision_unix":1779712630.1452901,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":13824,"affinity_cache_ratio":0.9726306902131852,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":13824,"estimated_new_tokens":389,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":584,"cache_hit":13824,"new_prefill":389,"score_linear":-13824.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":782,"cache_hit":0,"new_prefill":14213,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":14213,"score_linear":62414.0,"score_lmetric":57683},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29486,"ongoing_decode_tokens":29486,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":858,"cache_hit":0,"new_prefill":14213,"score_linear":29486.0,"score_lmetric":14213},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":0,"pending_prefill_tokens":68337,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":14213,"score_linear":81137.0,"score_lmetric":165100},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":14213,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18893,"ongoing_decode_tokens":18893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":14213,"score_linear":18893.0,"score_lmetric":14213},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105598,"ongoing_decode_tokens":105598,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":608,"cache_hit":0,"new_prefill":14213,"score_linear":105598.0,"score_lmetric":28426}],"chosen_score_linear":-13824.0,"chosen_score_lmetric":0,"t_first_token":281765.187946667,"t_first_token_unix":1779712630.20189,"t_done":281765.833907425,"t_done_unix":1779712630.8478508},{"request_id":"1294611:4:1300505:257","session_id":"1294611","input_length":24384,"t_proxy_recv":281766.150176072,"t_decision_unix":1779712631.1641176,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9868766404199475,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":320,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":584,"cache_hit":0,"new_prefill":24384,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":782,"cache_hit":24064,"new_prefill":320,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":24384,"score_linear":62414.0,"score_lmetric":67854},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":12800,"new_prefill":11584,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":0,"pending_prefill_tokens":68337,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":24384,"score_linear":81137.0,"score_lmetric":185442},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":24384,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18893,"ongoing_decode_tokens":18893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":24384,"score_linear":18893.0,"score_lmetric":24384},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60678,"ongoing_decode_tokens":60678,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":24384,"score_linear":60678.0,"score_lmetric":24384}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":281766.206349326,"t_first_token_unix":1779712631.2202926,"t_done":281766.570434142,"t_done_unix":1779712631.5843773},{"request_id":"1279412:8:1300001:255","session_id":"1279412","input_length":60678,"t_proxy_recv":281764.368794666,"t_decision_unix":1779712629.3827364,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":60416,"affinity_cache_ratio":0.9956821253172484,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":60416,"estimated_new_tokens":262,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":584,"cache_hit":0,"new_prefill":60678,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":782,"cache_hit":0,"new_prefill":60678,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":60678,"score_linear":62414.0,"score_lmetric":104148},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29486,"ongoing_decode_tokens":29486,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":858,"cache_hit":0,"new_prefill":60678,"score_linear":29486.0,"score_lmetric":60678},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":0,"pending_prefill_tokens":68337,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":60678,"score_linear":81137.0,"score_lmetric":258030},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":60678,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18893,"ongoing_decode_tokens":0,"pending_prefill_tokens":7117,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":60678,"score_linear":18893.0,"score_lmetric":67795},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":608,"cache_hit":60416,"new_prefill":262,"score_linear":-15496.0,"score_lmetric":262}],"chosen_score_linear":-15496.0,"chosen_score_lmetric":262,"t_first_token":281764.577477716,"t_first_token_unix":1779712629.5914214,"t_done":281766.628112536,"t_done_unix":1779712631.6420557},{"request_id":"1297473:1:1297473:239","session_id":"1297473","input_length":62414,"t_proxy_recv":281755.90883563,"t_decision_unix":1779712620.9227772,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":18944,"estimated_new_tokens":43470,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":557,"cache_hit":0,"new_prefill":62414,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":744,"cache_hit":0,"new_prefill":62414,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":856,"cache_hit":18944,"new_prefill":43470,"score_linear":-18944.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":836,"cache_hit":0,"new_prefill":62414,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":59507,"ongoing_decode_tokens":0,"pending_prefill_tokens":59507,"num_requests":1,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":62414,"score_linear":59507.0,"score_lmetric":121921},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":556,"cache_hit":0,"new_prefill":62414,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":730,"cache_hit":0,"new_prefill":62414,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105113,"ongoing_decode_tokens":44920,"pending_prefill_tokens":19745,"num_requests":2,"active_p_offloads":0,"cached_blocks":569,"cache_hit":0,"new_prefill":62414,"score_linear":105113.0,"score_lmetric":164318}],"chosen_score_linear":-18944.0,"chosen_score_lmetric":0,"t_first_token":281767.993833803,"t_first_token_unix":1779712633.007777,"t_done":281768.317933054,"t_done_unix":1779712633.3318768},{"request_id":"1294611:5:1301356:262","session_id":"1294611","input_length":25650,"t_proxy_recv":281768.970657068,"t_decision_unix":1779712633.9845986,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9381676413255361,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":1586,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":69555,"ongoing_decode_tokens":0,"pending_prefill_tokens":30643,"num_requests":2,"active_p_offloads":0,"cached_blocks":584,"cache_hit":0,"new_prefill":25650,"score_linear":69555.0,"score_lmetric":112586},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":782,"cache_hit":24064,"new_prefill":1586,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":940,"cache_hit":0,"new_prefill":25650,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":12800,"new_prefill":12850,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":81137,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":25650,"score_linear":81137.0,"score_lmetric":51300},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":25650,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":171155,"ongoing_decode_tokens":116704,"pending_prefill_tokens":3763,"num_requests":3,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":25650,"score_linear":171155.0,"score_lmetric":88239},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":25650,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":281769.258664337,"t_first_token_unix":1779712634.272608,"t_done":281769.566281283,"t_done_unix":1779712634.5802245},{"request_id":"1295690:2:1301706:264","session_id":"1295690","input_length":28972,"t_proxy_recv":281770.259302212,"t_decision_unix":1779712635.2732434,"policy":"unified","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":3,"affinity_cache_hit":10752,"affinity_cache_ratio":0.37111694049427035,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":28972,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":112736,"ongoing_decode_tokens":0,"pending_prefill_tokens":31840,"num_requests":3,"active_p_offloads":0,"cached_blocks":584,"cache_hit":0,"new_prefill":28972,"score_linear":112736.0,"score_lmetric":182436},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":785,"cache_hit":0,"new_prefill":28972,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":940,"cache_hit":0,"new_prefill":28972,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":28972,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":81137,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":10752,"new_prefill":18220,"score_linear":70385.0,"score_lmetric":36440},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":28972,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":171155,"ongoing_decode_tokens":171155,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":28972,"score_linear":171155.0,"score_lmetric":86916},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":28972,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_done":281770.264380949,"t_done_unix":1779712635.278327},{"request_id":"1297117:1:1297117:237","session_id":"1297117","input_length":59507,"t_proxy_recv":281754.639857251,"t_decision_unix":1779712619.6537983,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":59507,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":557,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":744,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":836,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":556,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50680,"ongoing_decode_tokens":50680,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":715,"cache_hit":0,"new_prefill":59507,"score_linear":50680.0,"score_lmetric":59507},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80589,"ongoing_decode_tokens":80589,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":568,"cache_hit":0,"new_prefill":59507,"score_linear":80589.0,"score_lmetric":119014}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281768.243738714,"t_first_token_unix":1779712633.257682,"t_done":281770.813219622,"t_done_unix":1779712635.8271632},{"request_id":"1268861:3:1298057:245","session_id":"1268861","input_length":21630,"t_proxy_recv":281757.686137193,"t_decision_unix":1779712622.7000785,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":12800,"affinity_cache_ratio":0.5917706888580675,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":8830,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13992,"ongoing_decode_tokens":0,"pending_prefill_tokens":12456,"num_requests":1,"active_p_offloads":0,"cached_blocks":557,"cache_hit":0,"new_prefill":21630,"score_linear":13992.0,"score_lmetric":34086},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23863,"ongoing_decode_tokens":23863,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":744,"cache_hit":0,"new_prefill":21630,"score_linear":23863.0,"score_lmetric":21630},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":21630,"score_linear":62414.0,"score_lmetric":65100},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29149,"ongoing_decode_tokens":29149,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":836,"cache_hit":0,"new_prefill":21630,"score_linear":29149.0,"score_lmetric":21630},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":59507,"ongoing_decode_tokens":0,"pending_prefill_tokens":59507,"num_requests":1,"active_p_offloads":0,"cached_blocks":579,"cache_hit":12800,"new_prefill":8830,"score_linear":46707.0,"score_lmetric":68337},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":556,"cache_hit":0,"new_prefill":21630,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":97145,"ongoing_decode_tokens":97145,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":730,"cache_hit":0,"new_prefill":21630,"score_linear":97145.0,"score_lmetric":21630},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105113,"ongoing_decode_tokens":44920,"pending_prefill_tokens":19745,"num_requests":2,"active_p_offloads":0,"cached_blocks":569,"cache_hit":0,"new_prefill":21630,"score_linear":105113.0,"score_lmetric":82750}],"chosen_score_linear":46707.0,"chosen_score_lmetric":68337,"t_first_token":281768.575204994,"t_first_token_unix":1779712633.5891488,"t_done":281771.619936068,"t_done_unix":1779712636.6338792},{"request_id":"1302071:1:1302071:267","session_id":"1302071","input_length":6527,"t_proxy_recv":281771.691232238,"t_decision_unix":1779712636.705174,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":6527,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":112736,"ongoing_decode_tokens":0,"pending_prefill_tokens":31840,"num_requests":3,"active_p_offloads":0,"cached_blocks":584,"cache_hit":0,"new_prefill":6527,"score_linear":112736.0,"score_lmetric":115101},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":785,"cache_hit":0,"new_prefill":6527,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":940,"cache_hit":0,"new_prefill":6527,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":6527,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":6527,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":34706,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":6527,"score_linear":34706.0,"score_lmetric":41233},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":171155,"ongoing_decode_tokens":171155,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":6527,"score_linear":171155.0,"score_lmetric":19581},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":6527,"score_linear":26441.0,"score_lmetric":32968}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281772.12550977,"t_first_token_unix":1779712637.139453,"t_done":281772.369557696,"t_done_unix":1779712637.3835008},{"request_id":"1302449:1:1302449:269","session_id":"1302449","input_length":1105,"t_proxy_recv":281773.225331621,"t_decision_unix":1779712638.239273,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1105,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":112736,"ongoing_decode_tokens":112736,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":584,"cache_hit":0,"new_prefill":1105,"score_linear":112736.0,"score_lmetric":3315},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":785,"cache_hit":0,"new_prefill":1105,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":940,"cache_hit":0,"new_prefill":1105,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":191,"num_requests":1,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":1105,"score_linear":49855.0,"score_lmetric":1296},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":1105,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":34706,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":1105,"score_linear":34706.0,"score_lmetric":35811},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":171155,"ongoing_decode_tokens":171155,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":1105,"score_linear":171155.0,"score_lmetric":3315},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":1105,"score_linear":26441.0,"score_lmetric":27546}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281773.357406262,"t_first_token_unix":1779712638.3713498,"t_done":281773.802640012,"t_done_unix":1779712638.8165832},{"request_id":"1302063:1:1302063:266","session_id":"1302063","input_length":26441,"t_proxy_recv":281771.677251905,"t_decision_unix":1779712636.6911933,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":26441,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":112736,"ongoing_decode_tokens":0,"pending_prefill_tokens":31840,"num_requests":3,"active_p_offloads":0,"cached_blocks":584,"cache_hit":0,"new_prefill":26441,"score_linear":112736.0,"score_lmetric":174843},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":785,"cache_hit":0,"new_prefill":26441,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":940,"cache_hit":0,"new_prefill":26441,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":26441,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":26441,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":34706,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":26441,"score_linear":34706.0,"score_lmetric":61147},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":171155,"ongoing_decode_tokens":171155,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":26441,"score_linear":171155.0,"score_lmetric":79323},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":26441,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281774.864565151,"t_first_token_unix":1779712639.8785083,"t_done":281775.118587778,"t_done_unix":1779712640.1325314},{"request_id":"1266668:11:1300918:260","session_id":"1266668","input_length":97811,"t_proxy_recv":281767.676453253,"t_decision_unix":1779712632.6903944,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":97280,"affinity_cache_ratio":0.9945711627526557,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":97280,"estimated_new_tokens":531,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":69555,"ongoing_decode_tokens":0,"pending_prefill_tokens":30643,"num_requests":2,"active_p_offloads":0,"cached_blocks":584,"cache_hit":4096,"new_prefill":93715,"score_linear":65459.0,"score_lmetric":248716},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":782,"cache_hit":0,"new_prefill":97811,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":97811,"score_linear":62414.0,"score_lmetric":141281},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":7680,"new_prefill":90131,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":0,"pending_prefill_tokens":68337,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":97811,"score_linear":81137.0,"score_lmetric":332296},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":97811,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18893,"ongoing_decode_tokens":18893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":97280,"new_prefill":531,"score_linear":-78387.0,"score_lmetric":531},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":637,"cache_hit":7680,"new_prefill":90131,"score_linear":-7680.0,"score_lmetric":0}],"chosen_score_linear":-78387.0,"chosen_score_lmetric":531,"t_first_token":281768.170720267,"t_first_token_unix":1779712633.1846635,"t_done":281775.441204493,"t_done_unix":1779712640.4551477},{"request_id":"1302907:1:1302907:272","session_id":"1302907","input_length":15958,"t_proxy_recv":281774.592793572,"t_decision_unix":1779712639.6067348,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":15958,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":112736,"ongoing_decode_tokens":112736,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":584,"cache_hit":0,"new_prefill":15958,"score_linear":112736.0,"score_lmetric":47874},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":30645,"ongoing_decode_tokens":30645,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":785,"cache_hit":0,"new_prefill":15958,"score_linear":30645.0,"score_lmetric":15958},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":15958,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":79803,"ongoing_decode_tokens":0,"pending_prefill_tokens":443,"num_requests":2,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":15958,"score_linear":79803.0,"score_lmetric":32802},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":15958,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":34706,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":15958,"score_linear":34706.0,"score_lmetric":50664},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":171155,"ongoing_decode_tokens":171155,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":15958,"score_linear":171155.0,"score_lmetric":47874},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":15958,"score_linear":26441.0,"score_lmetric":42399}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281776.047920624,"t_first_token_unix":1779712641.061864,"t_done":281776.248964736,"t_done_unix":1779712641.262908},{"request_id":"1299507:1:1299507:253","session_id":"1299507","input_length":18893,"t_proxy_recv":281762.565210545,"t_decision_unix":1779712627.579152,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":7117,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":584,"cache_hit":0,"new_prefill":18893,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":19157,"ongoing_decode_tokens":0,"pending_prefill_tokens":19157,"num_requests":1,"active_p_offloads":0,"cached_blocks":745,"cache_hit":0,"new_prefill":18893,"score_linear":19157.0,"score_lmetric":38050},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":18893,"score_linear":62414.0,"score_lmetric":62363},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":7963,"ongoing_decode_tokens":0,"pending_prefill_tokens":6427,"num_requests":1,"active_p_offloads":0,"cached_blocks":846,"cache_hit":0,"new_prefill":18893,"score_linear":7963.0,"score_lmetric":25320},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":0,"pending_prefill_tokens":68337,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":18893,"score_linear":81137.0,"score_lmetric":174460},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":8352,"ongoing_decode_tokens":0,"pending_prefill_tokens":8352,"num_requests":1,"active_p_offloads":0,"cached_blocks":574,"cache_hit":0,"new_prefill":18893,"score_linear":8352.0,"score_lmetric":27245},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":732,"cache_hit":11776,"new_prefill":7117,"score_linear":-11776.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105113,"ongoing_decode_tokens":105113,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":569,"cache_hit":0,"new_prefill":18893,"score_linear":105113.0,"score_lmetric":37786}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":0,"t_first_token":281764.476566891,"t_first_token_unix":1779712629.4905107,"t_done":281776.299151969,"t_done_unix":1779712641.313095},{"request_id":"1294611:6:1302460:270","session_id":"1294611","input_length":30645,"t_proxy_recv":281773.247798578,"t_decision_unix":1779712638.2617395,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":25600,"affinity_cache_ratio":0.8353728177516724,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":25600,"estimated_new_tokens":5045,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":112736,"ongoing_decode_tokens":112736,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":584,"cache_hit":0,"new_prefill":30645,"score_linear":112736.0,"score_lmetric":91935},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":785,"cache_hit":25600,"new_prefill":5045,"score_linear":-25600.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":1105,"ongoing_decode_tokens":0,"pending_prefill_tokens":1105,"num_requests":1,"active_p_offloads":0,"cached_blocks":940,"cache_hit":0,"new_prefill":30645,"score_linear":1105.0,"score_lmetric":31750},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":191,"num_requests":1,"active_p_offloads":0,"cached_blocks":871,"cache_hit":12800,"new_prefill":17845,"score_linear":37055.0,"score_lmetric":18036},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":30645,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":34706,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":30645,"score_linear":34706.0,"score_lmetric":65351},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":171155,"ongoing_decode_tokens":171155,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":30645,"score_linear":171155.0,"score_lmetric":91935},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":30645,"score_linear":26441.0,"score_lmetric":57086}],"chosen_score_linear":-25600.0,"chosen_score_lmetric":0,"t_first_token":281774.296095771,"t_first_token_unix":1779712639.310039,"t_done":281776.405650554,"t_done_unix":1779712641.4195938},{"request_id":"1290365:5:1300807:259","session_id":"1290365","input_length":25948,"t_proxy_recv":281767.326700842,"t_decision_unix":1779712632.340642,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":25088,"affinity_cache_ratio":0.9668567905040851,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":25088,"estimated_new_tokens":860,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":0,"pending_prefill_tokens":29783,"num_requests":1,"active_p_offloads":0,"cached_blocks":584,"cache_hit":25088,"new_prefill":860,"score_linear":18519.0,"score_lmetric":30643},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":782,"cache_hit":0,"new_prefill":25948,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":25948,"score_linear":62414.0,"score_lmetric":69418},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":25948,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":0,"pending_prefill_tokens":68337,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":25948,"score_linear":81137.0,"score_lmetric":188570},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":25948,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18893,"ongoing_decode_tokens":18893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":25948,"score_linear":18893.0,"score_lmetric":25948},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":25948,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":18519.0,"chosen_score_lmetric":30643,"t_first_token":281772.810223374,"t_first_token_unix":1779712637.8241663,"t_done":281777.142967803,"t_done_unix":1779712642.1569111},{"request_id":"1302449:2:1303563:277","session_id":"1302449","input_length":3709,"t_proxy_recv":281776.922155253,"t_decision_unix":1779712641.9360967,"policy":"unified","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":7,"affinity_cache_hit":1024,"affinity_cache_ratio":0.27608519816662175,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3709,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":112736,"ongoing_decode_tokens":112736,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":584,"cache_hit":0,"new_prefill":3709,"score_linear":112736.0,"score_lmetric":11127},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":3709,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14944,"ongoing_decode_tokens":0,"pending_prefill_tokens":14944,"num_requests":1,"active_p_offloads":0,"cached_blocks":973,"cache_hit":1024,"new_prefill":2685,"score_linear":13920.0,"score_lmetric":17629},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":79803,"ongoing_decode_tokens":0,"pending_prefill_tokens":443,"num_requests":2,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":3709,"score_linear":79803.0,"score_lmetric":8304},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":15081,"ongoing_decode_tokens":0,"pending_prefill_tokens":15081,"num_requests":1,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":3709,"score_linear":15081.0,"score_lmetric":18790},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":3709,"score_linear":34706.0,"score_lmetric":3709},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":3709,"score_linear":176419.0,"score_lmetric":55770},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":688,"cache_hit":0,"new_prefill":3709,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281777.150374054,"t_first_token_unix":1779712642.1643174,"t_done":281777.374838131,"t_done_unix":1779712642.3887813},{"request_id":"1303651:1:1303651:278","session_id":"1303651","input_length":3065,"t_proxy_recv":281777.22323398,"t_decision_unix":1779712642.2371752,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":3065,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":86788,"ongoing_decode_tokens":86788,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":586,"cache_hit":0,"new_prefill":3065,"score_linear":86788.0,"score_lmetric":6130},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":3065,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14944,"ongoing_decode_tokens":0,"pending_prefill_tokens":14944,"num_requests":1,"active_p_offloads":0,"cached_blocks":973,"cache_hit":0,"new_prefill":3065,"score_linear":14944.0,"score_lmetric":18009},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":79803,"ongoing_decode_tokens":0,"pending_prefill_tokens":443,"num_requests":2,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":3065,"score_linear":79803.0,"score_lmetric":7016},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":15081,"ongoing_decode_tokens":0,"pending_prefill_tokens":15081,"num_requests":1,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":3065,"score_linear":15081.0,"score_lmetric":18146},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":3065,"score_linear":34706.0,"score_lmetric":3065},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":3065,"score_linear":176419.0,"score_lmetric":54482},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":3709,"ongoing_decode_tokens":3709,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":688,"cache_hit":0,"new_prefill":3065,"score_linear":3709.0,"score_lmetric":3065}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281777.412250173,"t_first_token_unix":1779712642.4261935,"t_done":281777.641691571,"t_done_unix":1779712642.6556344},{"request_id":"1303462:1:1303462:274","session_id":"1303462","input_length":14944,"t_proxy_recv":281776.624188331,"t_decision_unix":1779712641.63813,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":14944,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":112736,"ongoing_decode_tokens":112736,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":584,"cache_hit":0,"new_prefill":14944,"score_linear":112736.0,"score_lmetric":44832},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":14944,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":973,"cache_hit":0,"new_prefill":14944,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":79803,"ongoing_decode_tokens":0,"pending_prefill_tokens":443,"num_requests":2,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":14944,"score_linear":79803.0,"score_lmetric":30774},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":14944,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":14944,"score_linear":34706.0,"score_lmetric":14944},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":54451,"ongoing_decode_tokens":54451,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":14944,"score_linear":54451.0,"score_lmetric":14944},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":688,"cache_hit":0,"new_prefill":14944,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281777.944590057,"t_first_token_unix":1779712642.9585333,"t_done":281778.242776555,"t_done_unix":1779712643.2567198},{"request_id":"1303517:1:1303517:275","session_id":"1303517","input_length":15081,"t_proxy_recv":281776.789275029,"t_decision_unix":1779712641.8032165,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":15081,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":112736,"ongoing_decode_tokens":112736,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":584,"cache_hit":0,"new_prefill":15081,"score_linear":112736.0,"score_lmetric":45243},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":15081,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14944,"ongoing_decode_tokens":0,"pending_prefill_tokens":14944,"num_requests":1,"active_p_offloads":0,"cached_blocks":973,"cache_hit":0,"new_prefill":15081,"score_linear":14944.0,"score_lmetric":30025},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":79803,"ongoing_decode_tokens":0,"pending_prefill_tokens":443,"num_requests":2,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":15081,"score_linear":79803.0,"score_lmetric":31048},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":15081,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":15081,"score_linear":34706.0,"score_lmetric":15081},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":54451,"ongoing_decode_tokens":54451,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":15081,"score_linear":54451.0,"score_lmetric":15081},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":688,"cache_hit":0,"new_prefill":15081,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281778.140345954,"t_first_token_unix":1779712643.1542892,"t_done":281778.479276771,"t_done_unix":1779712643.49322},{"request_id":"1302063:2:1303816:279","session_id":"1302063","input_length":30248,"t_proxy_recv":281777.897337705,"t_decision_unix":1779712642.9112792,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":26112,"affinity_cache_ratio":0.8632636868553293,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":26112,"estimated_new_tokens":4136,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":86788,"ongoing_decode_tokens":86788,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":586,"cache_hit":0,"new_prefill":30248,"score_linear":86788.0,"score_lmetric":60496},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":30248,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14944,"ongoing_decode_tokens":0,"pending_prefill_tokens":14944,"num_requests":1,"active_p_offloads":0,"cached_blocks":973,"cache_hit":0,"new_prefill":30248,"score_linear":14944.0,"score_lmetric":45192},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":79803,"ongoing_decode_tokens":0,"pending_prefill_tokens":443,"num_requests":2,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":30248,"score_linear":79803.0,"score_lmetric":61382},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":15081,"ongoing_decode_tokens":0,"pending_prefill_tokens":15081,"num_requests":1,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":30248,"score_linear":15081.0,"score_lmetric":45329},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":30248,"score_linear":34706.0,"score_lmetric":30248},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":30248,"score_linear":176419.0,"score_lmetric":108848},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":695,"cache_hit":26112,"new_prefill":4136,"score_linear":-26112.0,"score_lmetric":0}],"chosen_score_linear":-26112.0,"chosen_score_lmetric":0,"t_first_token":281778.75030077,"t_first_token_unix":1779712643.7642448,"t_done":281779.068021509,"t_done_unix":1779712644.081965},{"request_id":"1304239:1:1304239:280","session_id":"1304239","input_length":1300,"t_proxy_recv":281779.423292865,"t_decision_unix":1779712644.4372342,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1300,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":86788,"ongoing_decode_tokens":86788,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":586,"cache_hit":0,"new_prefill":1300,"score_linear":86788.0,"score_lmetric":2600},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":1300,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1002,"cache_hit":0,"new_prefill":1300,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":79803,"ongoing_decode_tokens":0,"pending_prefill_tokens":443,"num_requests":2,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":1300,"score_linear":79803.0,"score_lmetric":3486},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":741,"cache_hit":0,"new_prefill":1300,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":1300,"score_linear":34706.0,"score_lmetric":1300},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":1300,"score_linear":176419.0,"score_lmetric":50952},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":1300,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281779.520142092,"t_first_token_unix":1779712644.5340858,"t_done":281779.664286603,"t_done_unix":1779712644.67823},{"request_id":"1260327:8:1301686:263","session_id":"1260327","input_length":43181,"t_proxy_recv":281770.172745887,"t_decision_unix":1779712635.1866872,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":41984,"affinity_cache_ratio":0.9722794747689957,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":41984,"estimated_new_tokens":1197,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":69555,"ongoing_decode_tokens":0,"pending_prefill_tokens":30643,"num_requests":2,"active_p_offloads":0,"cached_blocks":584,"cache_hit":41984,"new_prefill":1197,"score_linear":27571.0,"score_lmetric":63680},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":785,"cache_hit":0,"new_prefill":43181,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":940,"cache_hit":0,"new_prefill":43181,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":43181,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":81137,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":43181,"score_linear":81137.0,"score_lmetric":86362},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":43181,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":171155,"ongoing_decode_tokens":171155,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":43181,"score_linear":171155.0,"score_lmetric":129543},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":637,"cache_hit":41984,"new_prefill":1197,"score_linear":-41984.0,"score_lmetric":0}],"chosen_score_linear":27571.0,"chosen_score_lmetric":63680,"t_first_token":281773.038706309,"t_first_token_unix":1779712638.0526497,"t_done":281780.61662897,"t_done_unix":1779712645.6305728},{"request_id":"1253743:3:1302294:268","session_id":"1253743","input_length":49855,"t_proxy_recv":281772.61053785,"t_decision_unix":1779712637.6244793,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":49664,"affinity_cache_ratio":0.996168889780363,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":191,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":112736,"ongoing_decode_tokens":0,"pending_prefill_tokens":31840,"num_requests":3,"active_p_offloads":0,"cached_blocks":584,"cache_hit":0,"new_prefill":49855,"score_linear":112736.0,"score_lmetric":245085},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":785,"cache_hit":4608,"new_prefill":45247,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":940,"cache_hit":0,"new_prefill":49855,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":871,"cache_hit":49664,"new_prefill":191,"score_linear":-49664.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":49855,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":34706,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":49855,"score_linear":34706.0,"score_lmetric":84561},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":171155,"ongoing_decode_tokens":171155,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":49855,"score_linear":171155.0,"score_lmetric":149565},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":49855,"score_linear":26441.0,"score_lmetric":76296}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":281780.308672292,"t_first_token_unix":1779712645.3226163,"t_done":281781.06665224,"t_done_unix":1779712646.0805962},{"request_id":"1304239:2:1304866:281","session_id":"1304239","input_length":7798,"t_proxy_recv":281781.573070483,"t_decision_unix":1779712646.5870116,"policy":"unified","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":1024,"affinity_cache_ratio":0.13131572197999486,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":1024,"estimated_new_tokens":6774,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":7798,"score_linear":43607.0,"score_lmetric":7798},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":803,"cache_hit":1024,"new_prefill":6774,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1002,"cache_hit":0,"new_prefill":7798,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29948,"ongoing_decode_tokens":29948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":7798,"score_linear":29948.0,"score_lmetric":7798},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":741,"cache_hit":0,"new_prefill":7798,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":7798,"score_linear":34706.0,"score_lmetric":7798},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":7798,"score_linear":176419.0,"score_lmetric":63948},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":7798,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-1024.0,"chosen_score_lmetric":0,"t_first_token":281782.055548364,"t_first_token_unix":1779712647.0694919,"t_done":281782.203232179,"t_done_unix":1779712647.2171757},{"request_id":"1304943:1:1304943:283","session_id":"1304943","input_length":2772,"t_proxy_recv":281781.824530042,"t_decision_unix":1779712646.838471,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":2772,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":2772,"score_linear":43607.0,"score_lmetric":2772},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":7798,"ongoing_decode_tokens":0,"pending_prefill_tokens":6774,"num_requests":1,"active_p_offloads":0,"cached_blocks":803,"cache_hit":0,"new_prefill":2772,"score_linear":7798.0,"score_lmetric":9546},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1002,"cache_hit":0,"new_prefill":2772,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29948,"ongoing_decode_tokens":29948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":2772,"score_linear":29948.0,"score_lmetric":2772},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":741,"cache_hit":0,"new_prefill":2772,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":2772,"score_linear":34706.0,"score_lmetric":2772},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":2772,"score_linear":176419.0,"score_lmetric":53896},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":2772,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281781.997969539,"t_first_token_unix":1779712647.0119135,"t_done":281782.299041019,"t_done_unix":1779712647.312985},{"request_id":"1294611:6:1305080:284","session_id":"1294611","input_length":30995,"t_proxy_recv":281782.334553281,"t_decision_unix":1779712647.3484948,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":30720,"affinity_cache_ratio":0.9911276012260042,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":30720,"estimated_new_tokens":275,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":30995,"score_linear":43607.0,"score_lmetric":30995},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":816,"cache_hit":30720,"new_prefill":275,"score_linear":-30720.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1002,"cache_hit":0,"new_prefill":30995,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29948,"ongoing_decode_tokens":29948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":871,"cache_hit":12800,"new_prefill":18195,"score_linear":17148.0,"score_lmetric":18195},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":746,"cache_hit":0,"new_prefill":30995,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":30995,"score_linear":34706.0,"score_lmetric":30995},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":30995,"score_linear":176419.0,"score_lmetric":110342},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":30995,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-30720.0,"chosen_score_lmetric":0,"t_first_token":281782.407395505,"t_first_token_unix":1779712647.4213388,"t_done":281783.721385261,"t_done_unix":1779712648.7353287},{"request_id":"1303651:2:1305707:287","session_id":"1303651","input_length":12795,"t_proxy_recv":281784.76867621,"t_decision_unix":1779712649.7826176,"policy":"unified","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":3072,"affinity_cache_ratio":0.24009378663540445,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":3072,"estimated_new_tokens":9723,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":12795,"score_linear":43607.0,"score_lmetric":12795},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":816,"cache_hit":3072,"new_prefill":9723,"score_linear":-3072.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1002,"cache_hit":0,"new_prefill":12795,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29948,"ongoing_decode_tokens":29948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":12795,"score_linear":29948.0,"score_lmetric":12795},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":746,"cache_hit":0,"new_prefill":12795,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":12795,"score_linear":34706.0,"score_lmetric":12795},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":12795,"score_linear":176419.0,"score_lmetric":73942},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":12795,"score_linear":137117.0,"score_lmetric":178992}],"chosen_score_linear":-3072.0,"chosen_score_lmetric":0,"t_first_token":281785.702637401,"t_first_token_unix":1779712650.7165809,"t_done":281785.820935851,"t_done_unix":1779712650.834879},{"request_id":"1303462:2:1306126:290","session_id":"1303462","input_length":18631,"t_proxy_recv":281785.994400881,"t_decision_unix":1779712651.008342,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":14848,"affinity_cache_ratio":0.7969513176963126,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":14848,"estimated_new_tokens":3783,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":18631,"score_linear":43607.0,"score_lmetric":18631},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":835,"cache_hit":0,"new_prefill":18631,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1002,"cache_hit":14848,"new_prefill":3783,"score_linear":-14848.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29948,"ongoing_decode_tokens":29948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":18631,"score_linear":29948.0,"score_lmetric":18631},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":14027,"ongoing_decode_tokens":0,"pending_prefill_tokens":14027,"num_requests":1,"active_p_offloads":0,"cached_blocks":746,"cache_hit":0,"new_prefill":18631,"score_linear":14027.0,"score_lmetric":32658},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":18631,"score_linear":34706.0,"score_lmetric":18631},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":195334,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24659,"num_requests":3,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":18631,"score_linear":195334.0,"score_lmetric":129870},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":18631,"score_linear":137117.0,"score_lmetric":190664}],"chosen_score_linear":-14848.0,"chosen_score_lmetric":0,"t_first_token":281786.544907078,"t_first_token_unix":1779712651.5588505,"t_done":281786.57862557,"t_done_unix":1779712651.5925686},{"request_id":"1306347:1:1306347:291","session_id":"1306347","input_length":897,"t_proxy_recv":281786.781744033,"t_decision_unix":1779712651.7956853,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":897,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":897,"score_linear":43607.0,"score_lmetric":897},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":835,"cache_hit":0,"new_prefill":897,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1009,"cache_hit":0,"new_prefill":897,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29948,"ongoing_decode_tokens":29948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":897,"score_linear":29948.0,"score_lmetric":897},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":14027,"ongoing_decode_tokens":14027,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":746,"cache_hit":0,"new_prefill":897,"score_linear":14027.0,"score_lmetric":897},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":897,"score_linear":34706.0,"score_lmetric":897},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":195334,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24659,"num_requests":3,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":897,"score_linear":195334.0,"score_lmetric":76668},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":897,"score_linear":137117.0,"score_lmetric":155196}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281786.861232041,"t_first_token_unix":1779712651.8751752,"t_done":281787.230885096,"t_done_unix":1779712652.2448282},{"request_id":"1305906:1:1305906:289","session_id":"1305906","input_length":14027,"t_proxy_recv":281785.381034667,"t_decision_unix":1779712650.394976,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":14027,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":14027,"score_linear":43607.0,"score_lmetric":14027},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":12795,"ongoing_decode_tokens":0,"pending_prefill_tokens":9723,"num_requests":1,"active_p_offloads":0,"cached_blocks":816,"cache_hit":0,"new_prefill":14027,"score_linear":12795.0,"score_lmetric":23750},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1002,"cache_hit":0,"new_prefill":14027,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29948,"ongoing_decode_tokens":29948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":14027,"score_linear":29948.0,"score_lmetric":14027},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":746,"cache_hit":0,"new_prefill":14027,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":14027,"score_linear":34706.0,"score_lmetric":14027},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":195334,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24659,"num_requests":3,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":14027,"score_linear":195334.0,"score_lmetric":116058},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":14027,"score_linear":137117.0,"score_lmetric":181456}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281786.598084819,"t_first_token_unix":1779712651.612028,"t_done":281787.644926361,"t_done_unix":1779712652.6588695},{"request_id":"1253743:4:1307058:295","session_id":"1253743","input_length":49975,"t_proxy_recv":281789.337931024,"t_decision_unix":1779712654.3518724,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9937768884442221,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":311,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":49975,"score_linear":43607.0,"score_lmetric":49975},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57218,"ongoing_decode_tokens":0,"pending_prefill_tokens":18818,"num_requests":2,"active_p_offloads":0,"cached_blocks":836,"cache_hit":4608,"new_prefill":45367,"score_linear":52610.0,"score_lmetric":128370},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1009,"cache_hit":0,"new_prefill":49975,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":61006,"ongoing_decode_tokens":61006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":871,"cache_hit":49664,"new_prefill":311,"score_linear":11342.0,"score_lmetric":622},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":773,"cache_hit":0,"new_prefill":49975,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":49975,"score_linear":34706.0,"score_lmetric":49975},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":195334,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24659,"num_requests":3,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":49975,"score_linear":195334.0,"score_lmetric":223902},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":49975,"score_linear":137117.0,"score_lmetric":253352}],"chosen_score_linear":11342.0,"chosen_score_lmetric":622,"t_first_token":281789.461155622,"t_first_token_unix":1779712654.4751015,"t_done":281790.211460503,"t_done_unix":1779712655.2254043},{"request_id":"1304239:3:1307007:294","session_id":"1304239","input_length":13840,"t_proxy_recv":281789.136403933,"t_decision_unix":1779712654.1503453,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":7680,"affinity_cache_ratio":0.5549132947976878,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":6160,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":13840,"score_linear":43607.0,"score_lmetric":13840},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":43378,"ongoing_decode_tokens":0,"pending_prefill_tokens":12658,"num_requests":1,"active_p_offloads":0,"cached_blocks":836,"cache_hit":7680,"new_prefill":6160,"score_linear":35698.0,"score_lmetric":18818},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1009,"cache_hit":0,"new_prefill":13840,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":61006,"ongoing_decode_tokens":61006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":13840,"score_linear":61006.0,"score_lmetric":27680},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":773,"cache_hit":0,"new_prefill":13840,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":13840,"score_linear":34706.0,"score_lmetric":13840},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":195334,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24659,"num_requests":3,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":13840,"score_linear":195334.0,"score_lmetric":115497},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":13840,"score_linear":137117.0,"score_lmetric":181082}],"chosen_score_linear":35698.0,"chosen_score_lmetric":18818,"t_first_token":281792.417014694,"t_first_token_unix":1779712657.430958,"t_done":281792.648576485,"t_done_unix":1779712657.6625197},{"request_id":"1277428:2:1306373:292","session_id":"1277428","input_length":31058,"t_proxy_recv":281786.892289031,"t_decision_unix":1779712651.9062302,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":27136,"affinity_cache_ratio":0.8737201365187713,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":27136,"estimated_new_tokens":3922,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":31058,"score_linear":43607.0,"score_lmetric":31058},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":897,"ongoing_decode_tokens":897,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":835,"cache_hit":4608,"new_prefill":26450,"score_linear":-3711.0,"score_lmetric":26450},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1009,"cache_hit":0,"new_prefill":31058,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29948,"ongoing_decode_tokens":29948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":871,"cache_hit":27136,"new_prefill":3922,"score_linear":2812.0,"score_lmetric":3922},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":14027,"ongoing_decode_tokens":14027,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":746,"cache_hit":0,"new_prefill":31058,"score_linear":14027.0,"score_lmetric":31058},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":31058,"score_linear":34706.0,"score_lmetric":31058},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":195334,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24659,"num_requests":3,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":31058,"score_linear":195334.0,"score_lmetric":167151},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":31058,"score_linear":137117.0,"score_lmetric":215518}],"chosen_score_linear":2812.0,"chosen_score_lmetric":3922,"t_first_token":281787.765838466,"t_first_token_unix":1779712652.7797818,"t_done":281792.707324953,"t_done_unix":1779712657.7212682},{"request_id":"1269373:9:1300958:261","session_id":"1269373","input_length":54451,"t_proxy_recv":281767.826385526,"t_decision_unix":1779712632.840327,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":50688,"affinity_cache_ratio":0.9308919946373804,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":50688,"estimated_new_tokens":3763,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":69555,"ongoing_decode_tokens":0,"pending_prefill_tokens":30643,"num_requests":2,"active_p_offloads":0,"cached_blocks":584,"cache_hit":4096,"new_prefill":50355,"score_linear":65459.0,"score_lmetric":161996},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":782,"cache_hit":0,"new_prefill":54451,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":54451,"score_linear":62414.0,"score_lmetric":97921},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":7680,"new_prefill":46771,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":0,"pending_prefill_tokens":68337,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":54451,"score_linear":81137.0,"score_lmetric":245576},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":54451,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":116704,"ongoing_decode_tokens":18893,"pending_prefill_tokens":531,"num_requests":2,"active_p_offloads":0,"cached_blocks":732,"cache_hit":50688,"new_prefill":3763,"score_linear":66016.0,"score_lmetric":8588},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":637,"cache_hit":7680,"new_prefill":46771,"score_linear":-7680.0,"score_lmetric":0}],"chosen_score_linear":66016.0,"chosen_score_lmetric":8588,"t_first_token":281769.333114203,"t_first_token_unix":1779712634.3470576,"t_done":281793.001583964,"t_done_unix":1779712658.0155272},{"request_id":"1303517:2:1307703:297","session_id":"1303517","input_length":23131,"t_proxy_recv":281791.662458032,"t_decision_unix":1779712656.676399,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":14848,"affinity_cache_ratio":0.6419091262807488,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":14848,"estimated_new_tokens":8283,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":23131,"score_linear":43607.0,"score_lmetric":23131},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57218,"ongoing_decode_tokens":0,"pending_prefill_tokens":18818,"num_requests":2,"active_p_offloads":0,"cached_blocks":836,"cache_hit":0,"new_prefill":23131,"score_linear":57218.0,"score_lmetric":83898},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1009,"cache_hit":0,"new_prefill":23131,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":61006,"ongoing_decode_tokens":61006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":23131,"score_linear":61006.0,"score_lmetric":46262},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":773,"cache_hit":14848,"new_prefill":8283,"score_linear":-14848.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":23131,"score_linear":34706.0,"score_lmetric":23131},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":195334,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24659,"num_requests":3,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":23131,"score_linear":195334.0,"score_lmetric":143370},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":23131,"score_linear":137117.0,"score_lmetric":199664}],"chosen_score_linear":-14848.0,"chosen_score_lmetric":0,"t_first_token":281792.921097695,"t_first_token_unix":1779712657.9350429,"t_done":281793.306301848,"t_done_unix":1779712658.3202455},{"request_id":"1302449:3:1308121:299","session_id":"1302449","input_length":8729,"t_proxy_recv":281793.070056399,"t_decision_unix":1779712658.0839982,"policy":"unified","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":2,"affinity_cache_hit":3584,"affinity_cache_ratio":0.41058540497193263,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":1024,"estimated_new_tokens":7705,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":8729,"score_linear":43607.0,"score_lmetric":8729},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":43378,"ongoing_decode_tokens":43378,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":848,"cache_hit":0,"new_prefill":8729,"score_linear":43378.0,"score_lmetric":8729},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1009,"cache_hit":1024,"new_prefill":7705,"score_linear":-1024.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29948,"ongoing_decode_tokens":29948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":8729,"score_linear":29948.0,"score_lmetric":8729},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":23131,"ongoing_decode_tokens":23131,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":773,"cache_hit":0,"new_prefill":8729,"score_linear":23131.0,"score_lmetric":8729},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":8729,"score_linear":34706.0,"score_lmetric":8729},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":200029,"ongoing_decode_tokens":140883,"pending_prefill_tokens":4362,"num_requests":3,"active_p_offloads":0,"cached_blocks":756,"cache_hit":0,"new_prefill":8729,"score_linear":200029.0,"score_lmetric":39273},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":3584,"new_prefill":5145,"score_linear":133533.0,"score_lmetric":163692}],"chosen_score_linear":-1024.0,"chosen_score_lmetric":0,"t_first_token":281793.639761353,"t_first_token_unix":1779712658.6537054,"t_done":281793.723469132,"t_done_unix":1779712658.7374122},{"request_id":"1301929:1:1301929:265","session_id":"1301929","input_length":34706,"t_proxy_recv":281771.127646713,"t_decision_unix":1779712636.141588,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":34706,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":112736,"ongoing_decode_tokens":0,"pending_prefill_tokens":31840,"num_requests":3,"active_p_offloads":0,"cached_blocks":584,"cache_hit":3584,"new_prefill":31122,"score_linear":109152.0,"score_lmetric":188886},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":785,"cache_hit":0,"new_prefill":34706,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":940,"cache_hit":0,"new_prefill":34706,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":34706,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21630,"ongoing_decode_tokens":21630,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":695,"cache_hit":0,"new_prefill":34706,"score_linear":21630.0,"score_lmetric":34706},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":34706,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":171155,"ongoing_decode_tokens":171155,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":34706,"score_linear":171155.0,"score_lmetric":104118},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":34706,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281776.146506818,"t_first_token_unix":1779712641.1604502,"t_done":281794.955954443,"t_done_unix":1779712659.9698977},{"request_id":"1303651:3:1308202:300","session_id":"1303651","input_length":22896,"t_proxy_recv":281793.325063332,"t_decision_unix":1779712658.3390048,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":12800,"affinity_cache_ratio":0.5590496156533893,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":10096,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":22896,"score_linear":43607.0,"score_lmetric":22896},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":43378,"ongoing_decode_tokens":43378,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":848,"cache_hit":12800,"new_prefill":10096,"score_linear":30578.0,"score_lmetric":10096},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":8729,"ongoing_decode_tokens":0,"pending_prefill_tokens":7705,"num_requests":1,"active_p_offloads":0,"cached_blocks":1009,"cache_hit":0,"new_prefill":22896,"score_linear":8729.0,"score_lmetric":30601},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29948,"ongoing_decode_tokens":29948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":22896,"score_linear":29948.0,"score_lmetric":22896},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":789,"cache_hit":0,"new_prefill":22896,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":22896,"score_linear":34706.0,"score_lmetric":22896},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":200029,"ongoing_decode_tokens":140883,"pending_prefill_tokens":4362,"num_requests":3,"active_p_offloads":0,"cached_blocks":756,"cache_hit":0,"new_prefill":22896,"score_linear":200029.0,"score_lmetric":81774},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":22896,"score_linear":137117.0,"score_lmetric":199194}],"chosen_score_linear":30578.0,"chosen_score_lmetric":10096,"t_first_token":281794.874707429,"t_first_token_unix":1779712659.888651,"t_done":281794.995989785,"t_done_unix":1779712660.0099328},{"request_id":"1294611:7:1306822:293","session_id":"1294611","input_length":43378,"t_proxy_recv":281788.603774511,"t_decision_unix":1779712653.617716,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":30720,"affinity_cache_ratio":0.7081930932730878,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":30720,"estimated_new_tokens":12658,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":43378,"score_linear":43607.0,"score_lmetric":43378},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":836,"cache_hit":30720,"new_prefill":12658,"score_linear":-30720.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1009,"cache_hit":0,"new_prefill":43378,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":61006,"ongoing_decode_tokens":61006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":871,"cache_hit":12800,"new_prefill":30578,"score_linear":48206.0,"score_lmetric":61156},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":773,"cache_hit":0,"new_prefill":43378,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":43378,"score_linear":34706.0,"score_lmetric":43378},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":195334,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24659,"num_requests":3,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":43378,"score_linear":195334.0,"score_lmetric":204111},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":43378,"score_linear":137117.0,"score_lmetric":240158}],"chosen_score_linear":-30720.0,"chosen_score_lmetric":0,"t_first_token":281791.827535623,"t_first_token_unix":1779712656.841479,"t_done":281795.092762341,"t_done_unix":1779712660.1067057},{"request_id":"1272313:11:1302571:271","session_id":"1272313","input_length":29948,"t_proxy_recv":281773.553507318,"t_decision_unix":1779712638.5674489,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":29696,"affinity_cache_ratio":0.991585414718846,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":29696,"estimated_new_tokens":252,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":112736,"ongoing_decode_tokens":112736,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":584,"cache_hit":4096,"new_prefill":25852,"score_linear":108640.0,"score_lmetric":77556},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":30645,"ongoing_decode_tokens":0,"pending_prefill_tokens":5045,"num_requests":1,"active_p_offloads":0,"cached_blocks":785,"cache_hit":0,"new_prefill":29948,"score_linear":30645.0,"score_lmetric":34993},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":1105,"ongoing_decode_tokens":1105,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":940,"cache_hit":0,"new_prefill":29948,"score_linear":1105.0,"score_lmetric":29948},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":191,"num_requests":1,"active_p_offloads":0,"cached_blocks":871,"cache_hit":29696,"new_prefill":252,"score_linear":20159.0,"score_lmetric":443},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":29948,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":34706,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":29948,"score_linear":34706.0,"score_lmetric":64654},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":171155,"ongoing_decode_tokens":171155,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":732,"cache_hit":10752,"new_prefill":19196,"score_linear":160403.0,"score_lmetric":57588},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":637,"cache_hit":7680,"new_prefill":22268,"score_linear":18761.0,"score_lmetric":48709}],"chosen_score_linear":20159.0,"chosen_score_lmetric":443,"t_first_token":281780.31016962,"t_first_token_unix":1779712645.3241127,"t_done":281795.163661942,"t_done_unix":1779712660.1776056},{"request_id":"1303517:3:1309213:303","session_id":"1303517","input_length":23259,"t_proxy_recv":281797.068376593,"t_decision_unix":1779712662.0823176,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9905842899522765,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":23040,"estimated_new_tokens":219,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":23259,"score_linear":43607.0,"score_lmetric":23259},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":43710,"ongoing_decode_tokens":43710,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":892,"cache_hit":0,"new_prefill":23259,"score_linear":43710.0,"score_lmetric":23259},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":23259,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":4679,"ongoing_decode_tokens":4679,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":881,"cache_hit":0,"new_prefill":23259,"score_linear":4679.0,"score_lmetric":23259},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":789,"cache_hit":23040,"new_prefill":219,"score_linear":-23040.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":659,"cache_hit":0,"new_prefill":23259,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":200029,"ongoing_decode_tokens":200029,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":756,"cache_hit":0,"new_prefill":23259,"score_linear":200029.0,"score_lmetric":69777},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":23259,"score_linear":137117.0,"score_lmetric":199920}],"chosen_score_linear":-23040.0,"chosen_score_lmetric":0,"t_first_token":281797.141442721,"t_first_token_unix":1779712662.1553867,"t_done":281797.194245755,"t_done_unix":1779712662.208189},{"request_id":"1309040:1:1309040:302","session_id":"1309040","input_length":4679,"t_proxy_recv":281796.441095791,"t_decision_unix":1779712661.4550369,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4679,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":4679,"score_linear":43607.0,"score_lmetric":4679},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":43710,"ongoing_decode_tokens":43710,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":892,"cache_hit":0,"new_prefill":4679,"score_linear":43710.0,"score_lmetric":4679},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":4679,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":881,"cache_hit":0,"new_prefill":4679,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":789,"cache_hit":0,"new_prefill":4679,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":659,"cache_hit":0,"new_prefill":4679,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":200029,"ongoing_decode_tokens":200029,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":756,"cache_hit":0,"new_prefill":4679,"score_linear":200029.0,"score_lmetric":14037},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":4679,"score_linear":137117.0,"score_lmetric":162760}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281796.735237117,"t_first_token_unix":1779712661.7491806,"t_done":281797.23455559,"t_done_unix":1779712662.248499},{"request_id":"1294611:8:1308970:301","session_id":"1294611","input_length":43710,"t_proxy_recv":281796.112358462,"t_decision_unix":1779712661.1262994,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":43520,"affinity_cache_ratio":0.9956531686113017,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":43520,"estimated_new_tokens":190,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":43710,"score_linear":43607.0,"score_lmetric":43710},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":892,"cache_hit":43520,"new_prefill":190,"score_linear":-43520.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":43710,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":881,"cache_hit":12800,"new_prefill":30910,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":789,"cache_hit":0,"new_prefill":43710,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":659,"cache_hit":0,"new_prefill":43710,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":200029,"ongoing_decode_tokens":200029,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":756,"cache_hit":0,"new_prefill":43710,"score_linear":200029.0,"score_lmetric":131130},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":43710,"score_linear":137117.0,"score_lmetric":240822}],"chosen_score_linear":-43520.0,"chosen_score_lmetric":0,"t_first_token":281796.276376915,"t_first_token_unix":1779712661.2903204,"t_done":281797.547360914,"t_done_unix":1779712662.5613046},{"request_id":"1299507:2:1305770:288","session_id":"1299507","input_length":18915,"t_proxy_recv":281784.961894289,"t_decision_unix":1779712649.9758353,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":18432,"affinity_cache_ratio":0.9744647105471848,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":18432,"estimated_new_tokens":483,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":18915,"score_linear":43607.0,"score_lmetric":18915},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":12795,"ongoing_decode_tokens":0,"pending_prefill_tokens":9723,"num_requests":1,"active_p_offloads":0,"cached_blocks":816,"cache_hit":0,"new_prefill":18915,"score_linear":12795.0,"score_lmetric":28638},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1002,"cache_hit":0,"new_prefill":18915,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29948,"ongoing_decode_tokens":29948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":18915,"score_linear":29948.0,"score_lmetric":18915},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":746,"cache_hit":0,"new_prefill":18915,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":18915,"score_linear":34706.0,"score_lmetric":18915},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":748,"cache_hit":18432,"new_prefill":483,"score_linear":157987.0,"score_lmetric":49318},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":18915,"score_linear":137117.0,"score_lmetric":191232}],"chosen_score_linear":157987.0,"chosen_score_lmetric":49318,"t_first_token":281792.939258578,"t_first_token_unix":1779712657.9532018,"t_done":281798.346927693,"t_done_unix":1779712663.3608713},{"request_id":"1269373:10:1304899:282","session_id":"1269373","input_length":59146,"t_proxy_recv":281793.014666352,"t_decision_unix":1779712658.0286078,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9262502958779968,"affinity_num_requests":2,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":54784,"estimated_new_tokens":4362,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":4096,"new_prefill":55050,"score_linear":39511.0,"score_lmetric":55050},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":43378,"ongoing_decode_tokens":43378,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":848,"cache_hit":0,"new_prefill":59146,"score_linear":43378.0,"score_lmetric":59146},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1009,"cache_hit":0,"new_prefill":59146,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29948,"ongoing_decode_tokens":29948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":7680,"new_prefill":51466,"score_linear":22268.0,"score_lmetric":51466},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":23131,"ongoing_decode_tokens":23131,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":773,"cache_hit":0,"new_prefill":59146,"score_linear":23131.0,"score_lmetric":59146},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":59146,"score_linear":34706.0,"score_lmetric":59146},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":140883,"ongoing_decode_tokens":140883,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":756,"cache_hit":54784,"new_prefill":4362,"score_linear":86099.0,"score_lmetric":8724},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":7680,"new_prefill":51466,"score_linear":129437.0,"score_lmetric":256334}],"chosen_score_linear":86099.0,"chosen_score_lmetric":8724,"t_first_token":281794.656138318,"t_first_token_unix":1779712659.670082,"t_done":281798.78385439,"t_done_unix":1779712663.797798},{"request_id":"1309252:1:1309252:304","session_id":"1309252","input_length":24545,"t_proxy_recv":281797.272116245,"t_decision_unix":1779712662.2860575,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":24545,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":24545,"score_linear":43607.0,"score_lmetric":24545},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":43710,"ongoing_decode_tokens":43710,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":892,"cache_hit":0,"new_prefill":24545,"score_linear":43710.0,"score_lmetric":24545},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":24545,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":890,"cache_hit":0,"new_prefill":24545,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":789,"cache_hit":0,"new_prefill":24545,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":659,"cache_hit":0,"new_prefill":24545,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":200029,"ongoing_decode_tokens":200029,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":756,"cache_hit":0,"new_prefill":24545,"score_linear":200029.0,"score_lmetric":73635},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":24545,"score_linear":137117.0,"score_lmetric":202492}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281800.11640976,"t_first_token_unix":1779712665.1303544,"t_done":281801.996417108,"t_done_unix":1779712667.01036},{"request_id":"1310590:1:1310590:306","session_id":"1310590","input_length":18089,"t_proxy_recv":281801.854685689,"t_decision_unix":1779712666.8686266,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":18089,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":18089,"score_linear":43607.0,"score_lmetric":18089},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":892,"cache_hit":0,"new_prefill":18089,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":18089,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":890,"cache_hit":0,"new_prefill":18089,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":24545,"ongoing_decode_tokens":24545,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":789,"cache_hit":0,"new_prefill":18089,"score_linear":24545.0,"score_lmetric":18089},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":659,"cache_hit":0,"new_prefill":18089,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":764,"cache_hit":0,"new_prefill":18089,"score_linear":181414.0,"score_lmetric":36178},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":18089,"score_linear":137117.0,"score_lmetric":189580}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281803.640364687,"t_first_token_unix":1779712668.6543078,"t_done":281804.132772956,"t_done_unix":1779712669.1467164},{"request_id":"1305420:1:1305420:285","session_id":"1305420","input_length":75826,"t_proxy_recv":281783.592271981,"t_decision_unix":1779712648.6062129,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":75826,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":75826,"score_linear":43607.0,"score_lmetric":75826},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":30995,"ongoing_decode_tokens":30995,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":816,"cache_hit":0,"new_prefill":75826,"score_linear":30995.0,"score_lmetric":75826},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1002,"cache_hit":0,"new_prefill":75826,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29948,"ongoing_decode_tokens":29948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":75826,"score_linear":29948.0,"score_lmetric":75826},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":746,"cache_hit":0,"new_prefill":75826,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":75826,"score_linear":34706.0,"score_lmetric":75826},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":75826,"score_linear":176419.0,"score_lmetric":200004},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":75826,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281803.473538206,"t_first_token_unix":1779712668.487482,"t_done":281804.365343164,"t_done_unix":1779712669.3792865},{"request_id":"1279412:9:1305633:286","session_id":"1279412","input_length":61291,"t_proxy_recv":281784.505337725,"t_decision_unix":1779712649.5192785,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":60416,"affinity_cache_ratio":0.985723841999641,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":60416,"estimated_new_tokens":875,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":61291,"score_linear":43607.0,"score_lmetric":61291},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":816,"cache_hit":0,"new_prefill":61291,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1002,"cache_hit":0,"new_prefill":61291,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":29948,"ongoing_decode_tokens":29948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":61291,"score_linear":29948.0,"score_lmetric":61291},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":746,"cache_hit":0,"new_prefill":61291,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":61291,"score_linear":34706.0,"score_lmetric":61291},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":54451,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":748,"cache_hit":0,"new_prefill":61291,"score_linear":176419.0,"score_lmetric":170934},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":703,"cache_hit":60416,"new_prefill":875,"score_linear":15410.0,"score_lmetric":76701}],"chosen_score_linear":15410.0,"chosen_score_lmetric":76701,"t_first_token":281803.480942566,"t_first_token_unix":1779712668.4948864,"t_done":281804.64739802,"t_done_unix":1779712669.6613414},{"request_id":"1279412:10:1308091:298","session_id":"1279412","input_length":61505,"t_proxy_recv":281804.65788028,"t_decision_unix":1779712669.6718218,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":60928,"affinity_cache_ratio":0.9906186488903341,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":60928,"estimated_new_tokens":577,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":61505,"score_linear":43607.0,"score_lmetric":61505},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":892,"cache_hit":0,"new_prefill":61505,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":61505,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":890,"cache_hit":0,"new_prefill":61505,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":37802,"ongoing_decode_tokens":0,"pending_prefill_tokens":12202,"num_requests":2,"active_p_offloads":0,"cached_blocks":837,"cache_hit":0,"new_prefill":61505,"score_linear":37802.0,"score_lmetric":147414},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":694,"cache_hit":0,"new_prefill":61505,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":764,"cache_hit":0,"new_prefill":61505,"score_linear":181414.0,"score_lmetric":123010},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":852,"cache_hit":60928,"new_prefill":577,"score_linear":-60928.0,"score_lmetric":0}],"chosen_score_linear":-60928.0,"chosen_score_lmetric":0,"t_first_token":281804.822377168,"t_first_token_unix":1779712669.8363202,"t_done":281805.597376095,"t_done_unix":1779712670.6113195},{"request_id":"1277428:3:1311644:311","session_id":"1277428","input_length":31611,"t_proxy_recv":281805.621452078,"t_decision_unix":1779712670.6353939,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9880105026731201,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":31232,"estimated_new_tokens":379,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":31611,"score_linear":43607.0,"score_lmetric":31611},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":892,"cache_hit":4608,"new_prefill":27003,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":31611,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":8996,"ongoing_decode_tokens":0,"pending_prefill_tokens":292,"num_requests":1,"active_p_offloads":0,"cached_blocks":890,"cache_hit":31232,"new_prefill":379,"score_linear":-22236.0,"score_lmetric":671},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82594,"ongoing_decode_tokens":37802,"pending_prefill_tokens":20216,"num_requests":3,"active_p_offloads":0,"cached_blocks":837,"cache_hit":0,"new_prefill":31611,"score_linear":82594.0,"score_lmetric":155481},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":694,"cache_hit":0,"new_prefill":31611,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":764,"cache_hit":0,"new_prefill":31611,"score_linear":181414.0,"score_lmetric":63222},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64558,"ongoing_decode_tokens":0,"pending_prefill_tokens":3118,"num_requests":1,"active_p_offloads":0,"cached_blocks":853,"cache_hit":0,"new_prefill":31611,"score_linear":64558.0,"score_lmetric":34729}],"chosen_score_linear":-22236.0,"chosen_score_lmetric":671,"t_first_token":281806.146752856,"t_first_token_unix":1779712671.160696,"t_done":281806.549546015,"t_done_unix":1779712671.5634892},{"request_id":"1269373:11:1307381:296","session_id":"1269373","input_length":59446,"t_proxy_recv":281798.799795173,"t_decision_unix":1779712663.8137343,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9904787538270027,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":566,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":4096,"new_prefill":55350,"score_linear":39511.0,"score_lmetric":55350},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":892,"cache_hit":0,"new_prefill":59446,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":59446,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":890,"cache_hit":7680,"new_prefill":51766,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":24545,"ongoing_decode_tokens":0,"pending_prefill_tokens":24545,"num_requests":1,"active_p_offloads":0,"cached_blocks":789,"cache_hit":0,"new_prefill":59446,"score_linear":24545.0,"score_lmetric":83991},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":659,"cache_hit":0,"new_prefill":59446,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":764,"cache_hit":58880,"new_prefill":566,"score_linear":63088.0,"score_lmetric":566},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":0,"pending_prefill_tokens":76701,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":7680,"new_prefill":51766,"score_linear":129437.0,"score_lmetric":256934}],"chosen_score_linear":63088.0,"chosen_score_lmetric":566,"t_first_token":281798.962091661,"t_first_token_unix":1779712663.976035,"t_done":281806.875650723,"t_done_unix":1779712671.8895948},{"request_id":"1297811:2:1300578:258","session_id":"1297811","input_length":43607,"t_proxy_recv":281766.470881148,"t_decision_unix":1779712631.4848223,"policy":"unified","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":13824,"affinity_cache_ratio":0.3170133235489715,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":13824,"estimated_new_tokens":29783,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":584,"cache_hit":13824,"new_prefill":29783,"score_linear":-13824.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":24384,"ongoing_decode_tokens":24384,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":782,"cache_hit":0,"new_prefill":43607,"score_linear":24384.0,"score_lmetric":43607},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":43607,"score_linear":62414.0,"score_lmetric":87077},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":43607,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81137,"ongoing_decode_tokens":0,"pending_prefill_tokens":68337,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":43607,"score_linear":81137.0,"score_lmetric":223888},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":43607,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18893,"ongoing_decode_tokens":18893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":43607,"score_linear":18893.0,"score_lmetric":43607},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60678,"ongoing_decode_tokens":60678,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":43607,"score_linear":60678.0,"score_lmetric":43607}],"chosen_score_linear":-13824.0,"chosen_score_lmetric":0,"t_first_token":281772.806515451,"t_first_token_unix":1779712637.8204584,"t_done":281807.490043523,"t_done_unix":1779712672.5039873},{"request_id":"1258908:2:1311606:310","session_id":"1258908","input_length":8996,"t_proxy_recv":281805.429839854,"t_decision_unix":1779712670.4437814,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":8704,"affinity_cache_ratio":0.9675411293908404,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":8704,"estimated_new_tokens":292,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":8996,"score_linear":43607.0,"score_lmetric":8996},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":892,"cache_hit":0,"new_prefill":8996,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":8996,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":890,"cache_hit":8704,"new_prefill":292,"score_linear":-8704.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82594,"ongoing_decode_tokens":37802,"pending_prefill_tokens":20216,"num_requests":3,"active_p_offloads":0,"cached_blocks":837,"cache_hit":0,"new_prefill":8996,"score_linear":82594.0,"score_lmetric":87636},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":694,"cache_hit":0,"new_prefill":8996,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":764,"cache_hit":0,"new_prefill":8996,"score_linear":181414.0,"score_lmetric":17992},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":61505,"ongoing_decode_tokens":61505,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":8996,"score_linear":61505.0,"score_lmetric":8996}],"chosen_score_linear":-8704.0,"chosen_score_lmetric":0,"t_first_token":281806.101929046,"t_first_token_unix":1779712671.1158726,"t_done":281808.165835221,"t_done_unix":1779712673.1797786},{"request_id":"1311753:1:1311753:312","session_id":"1311753","input_length":20175,"t_proxy_recv":281806.002186179,"t_decision_unix":1779712671.0161273,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":20175,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":20175,"score_linear":43607.0,"score_lmetric":20175},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":892,"cache_hit":0,"new_prefill":20175,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":20175,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":40607,"ongoing_decode_tokens":0,"pending_prefill_tokens":671,"num_requests":2,"active_p_offloads":0,"cached_blocks":890,"cache_hit":0,"new_prefill":20175,"score_linear":40607.0,"score_lmetric":41692},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82594,"ongoing_decode_tokens":37802,"pending_prefill_tokens":20216,"num_requests":3,"active_p_offloads":0,"cached_blocks":837,"cache_hit":0,"new_prefill":20175,"score_linear":82594.0,"score_lmetric":121173},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":694,"cache_hit":0,"new_prefill":20175,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":764,"cache_hit":0,"new_prefill":20175,"score_linear":181414.0,"score_lmetric":40350},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64558,"ongoing_decode_tokens":0,"pending_prefill_tokens":3118,"num_requests":1,"active_p_offloads":0,"cached_blocks":853,"cache_hit":0,"new_prefill":20175,"score_linear":64558.0,"score_lmetric":23293}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281808.098258936,"t_first_token_unix":1779712673.1122024,"t_done":281808.278669914,"t_done_unix":1779712673.292613},{"request_id":"1279412:11:1310014:305","session_id":"1279412","input_length":64558,"t_proxy_recv":281805.613661069,"t_decision_unix":1779712670.6276023,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":61440,"affinity_cache_ratio":0.9517023451779795,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":61440,"estimated_new_tokens":3118,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":64558,"score_linear":43607.0,"score_lmetric":64558},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":892,"cache_hit":0,"new_prefill":64558,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":64558,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":8996,"ongoing_decode_tokens":0,"pending_prefill_tokens":292,"num_requests":1,"active_p_offloads":0,"cached_blocks":890,"cache_hit":0,"new_prefill":64558,"score_linear":8996.0,"score_lmetric":64850},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82594,"ongoing_decode_tokens":37802,"pending_prefill_tokens":20216,"num_requests":3,"active_p_offloads":0,"cached_blocks":837,"cache_hit":0,"new_prefill":64558,"score_linear":82594.0,"score_lmetric":254322},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":694,"cache_hit":0,"new_prefill":64558,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":764,"cache_hit":0,"new_prefill":64558,"score_linear":181414.0,"score_lmetric":129116},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":853,"cache_hit":61440,"new_prefill":3118,"score_linear":-61440.0,"score_lmetric":0}],"chosen_score_linear":-61440.0,"chosen_score_lmetric":0,"t_first_token":281806.923341648,"t_first_token_unix":1779712671.937285,"t_done":281808.310972623,"t_done_unix":1779712673.3249156},{"request_id":"1294611:9:1312398:313","session_id":"1294611","input_length":44335,"t_proxy_recv":281808.287149866,"t_decision_unix":1779712673.3010914,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":43520,"affinity_cache_ratio":0.9816172324348709,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":43520,"estimated_new_tokens":815,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":649,"cache_hit":0,"new_prefill":44335,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":892,"cache_hit":43520,"new_prefill":815,"score_linear":-43520.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":44335,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":891,"cache_hit":12800,"new_prefill":31535,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82594,"ongoing_decode_tokens":37802,"pending_prefill_tokens":20216,"num_requests":3,"active_p_offloads":0,"cached_blocks":837,"cache_hit":0,"new_prefill":44335,"score_linear":82594.0,"score_lmetric":193653},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":733,"cache_hit":0,"new_prefill":44335,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":766,"cache_hit":0,"new_prefill":44335,"score_linear":121968.0,"score_lmetric":44335},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64558,"ongoing_decode_tokens":64558,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":853,"cache_hit":0,"new_prefill":44335,"score_linear":64558.0,"score_lmetric":44335}],"chosen_score_linear":-43520.0,"chosen_score_lmetric":0,"t_first_token":281808.545908108,"t_first_token_unix":1779712673.559852,"t_done":281809.838368572,"t_done_unix":1779712674.8523116},{"request_id":"1304943:2:1311189:307","session_id":"1304943","input_length":13164,"t_proxy_recv":281803.986664203,"t_decision_unix":1779712669.0006056,"policy":"unified","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":2560,"affinity_cache_ratio":0.19446976602856275,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":2560,"estimated_new_tokens":10604,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":13164,"score_linear":43607.0,"score_lmetric":13164},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":892,"cache_hit":0,"new_prefill":13164,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":13164,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":890,"cache_hit":0,"new_prefill":13164,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":837,"cache_hit":2560,"new_prefill":10604,"score_linear":-2560.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":18089,"ongoing_decode_tokens":18089,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":659,"cache_hit":0,"new_prefill":13164,"score_linear":18089.0,"score_lmetric":13164},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":764,"cache_hit":0,"new_prefill":13164,"score_linear":181414.0,"score_lmetric":26328},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":137117,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":13164,"score_linear":137117.0,"score_lmetric":26328}],"chosen_score_linear":-2560.0,"chosen_score_lmetric":0,"t_first_token":281805.033116945,"t_first_token_unix":1779712670.0470603,"t_done":281810.19446207,"t_done_unix":1779712675.2084053},{"request_id":"1303517:4:1311293:308","session_id":"1303517","input_length":24638,"t_proxy_recv":281804.302246429,"t_decision_unix":1779712669.3161874,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9351408393538436,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":23040,"estimated_new_tokens":1598,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":24638,"score_linear":43607.0,"score_lmetric":24638},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":892,"cache_hit":0,"new_prefill":24638,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":24638,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":890,"cache_hit":0,"new_prefill":24638,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":13164,"ongoing_decode_tokens":0,"pending_prefill_tokens":10604,"num_requests":1,"active_p_offloads":0,"cached_blocks":837,"cache_hit":23040,"new_prefill":1598,"score_linear":-9876.0,"score_lmetric":12202},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":694,"cache_hit":0,"new_prefill":24638,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":764,"cache_hit":0,"new_prefill":24638,"score_linear":181414.0,"score_lmetric":49276},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":137117,"ongoing_decode_tokens":137117,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":24638,"score_linear":137117.0,"score_lmetric":49276}],"chosen_score_linear":-9876.0,"chosen_score_lmetric":12202,"t_first_token":281805.2870113,"t_first_token_unix":1779712670.3009546,"t_done":281810.381413547,"t_done_unix":1779712675.3953567},{"request_id":"1303517:5:1313040:317","session_id":"1303517","input_length":25637,"t_proxy_recv":281810.396893185,"t_decision_unix":1779712675.4108346,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":24576,"affinity_cache_ratio":0.9586145024768888,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24576,"estimated_new_tokens":1061,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13360,"ongoing_decode_tokens":13360,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":649,"cache_hit":0,"new_prefill":25637,"score_linear":13360.0,"score_lmetric":25637},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":893,"cache_hit":0,"new_prefill":25637,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":25637,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":25637,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":44792,"ongoing_decode_tokens":44792,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":860,"cache_hit":24576,"new_prefill":1061,"score_linear":20216.0,"score_lmetric":1061},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26885,"ongoing_decode_tokens":0,"pending_prefill_tokens":26885,"num_requests":1,"active_p_offloads":0,"cached_blocks":733,"cache_hit":0,"new_prefill":25637,"score_linear":26885.0,"score_lmetric":52522},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":184808,"ongoing_decode_tokens":121968,"pending_prefill_tokens":2936,"num_requests":2,"active_p_offloads":0,"cached_blocks":766,"cache_hit":0,"new_prefill":25637,"score_linear":184808.0,"score_lmetric":57146},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":25637,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":20216.0,"chosen_score_lmetric":1061,"t_first_token":281810.678132207,"t_first_token_unix":1779712675.6920755,"t_done":281810.732751785,"t_done_unix":1779712675.7466948},{"request_id":"1306347:2:1312642:314","session_id":"1306347","input_length":13360,"t_proxy_recv":281809.177440358,"t_decision_unix":1779712674.1913817,"policy":"unified","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":0,"affinity_cache_hit":512,"affinity_cache_ratio":0.03832335329341317,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":13360,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":649,"cache_hit":0,"new_prefill":13360,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":44335,"ongoing_decode_tokens":44335,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":892,"cache_hit":512,"new_prefill":12848,"score_linear":43823.0,"score_lmetric":12848},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":13360,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":13360,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82594,"ongoing_decode_tokens":37802,"pending_prefill_tokens":20216,"num_requests":3,"active_p_offloads":0,"cached_blocks":837,"cache_hit":0,"new_prefill":13360,"score_linear":82594.0,"score_lmetric":100728},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":733,"cache_hit":0,"new_prefill":13360,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":766,"cache_hit":0,"new_prefill":13360,"score_linear":121968.0,"score_lmetric":13360},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":13360,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281810.31258347,"t_first_token_unix":1779712675.3265266,"t_done":281810.862312519,"t_done_unix":1779712675.876256},{"request_id":"1309252:2:1311531:309","session_id":"1309252","input_length":44792,"t_proxy_recv":281805.200873953,"t_decision_unix":1779712670.2148151,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":24576,"affinity_cache_ratio":0.5486694052509377,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24576,"estimated_new_tokens":20216,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":44792,"score_linear":43607.0,"score_lmetric":44792},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":892,"cache_hit":0,"new_prefill":44792,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":44792,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":890,"cache_hit":0,"new_prefill":44792,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":37802,"ongoing_decode_tokens":13164,"pending_prefill_tokens":1598,"num_requests":2,"active_p_offloads":0,"cached_blocks":837,"cache_hit":24576,"new_prefill":20216,"score_linear":13226.0,"score_lmetric":43628},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":694,"cache_hit":0,"new_prefill":44792,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":764,"cache_hit":0,"new_prefill":44792,"score_linear":181414.0,"score_lmetric":89584},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":61505,"ongoing_decode_tokens":61505,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":44792,"score_linear":61505.0,"score_lmetric":44792}],"chosen_score_linear":13226.0,"chosen_score_lmetric":43628,"t_first_token":281810.160545469,"t_first_token_unix":1779712675.174489,"t_done":281811.147344105,"t_done_unix":1779712676.1612878},{"request_id":"1313181:1:1313181:318","session_id":"1313181","input_length":22686,"t_proxy_recv":281810.965823002,"t_decision_unix":1779712675.9797645,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":9886,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":22686,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":893,"cache_hit":0,"new_prefill":22686,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":22686,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":22686,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":44792,"ongoing_decode_tokens":44792,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":862,"cache_hit":0,"new_prefill":22686,"score_linear":44792.0,"score_lmetric":22686},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26885,"ongoing_decode_tokens":0,"pending_prefill_tokens":26885,"num_requests":1,"active_p_offloads":0,"cached_blocks":733,"cache_hit":0,"new_prefill":22686,"score_linear":26885.0,"score_lmetric":49571},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":184808,"ongoing_decode_tokens":121968,"pending_prefill_tokens":2936,"num_requests":2,"active_p_offloads":0,"cached_blocks":766,"cache_hit":0,"new_prefill":22686,"score_linear":184808.0,"score_lmetric":51244},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":12800,"new_prefill":9886,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-12800.0,"chosen_score_lmetric":0,"t_first_token":281812.480640358,"t_first_token_unix":1779712677.4945838,"t_done":281813.64224284,"t_done_unix":1779712678.6561859},{"request_id":"1268861:4:1312689:315","session_id":"1268861","input_length":26885,"t_proxy_recv":281809.345699177,"t_decision_unix":1779712674.3596406,"policy":"unified","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":5,"affinity_cache_hit":21504,"affinity_cache_ratio":0.7998512181513855,"affinity_num_requests":3,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":26885,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13360,"ongoing_decode_tokens":0,"pending_prefill_tokens":13360,"num_requests":1,"active_p_offloads":0,"cached_blocks":649,"cache_hit":0,"new_prefill":26885,"score_linear":13360.0,"score_lmetric":40245},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":44335,"ongoing_decode_tokens":44335,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":892,"cache_hit":0,"new_prefill":26885,"score_linear":44335.0,"score_lmetric":26885},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":26885,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":26885,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82594,"ongoing_decode_tokens":37802,"pending_prefill_tokens":20216,"num_requests":3,"active_p_offloads":0,"cached_blocks":837,"cache_hit":21504,"new_prefill":5381,"score_linear":61090.0,"score_lmetric":76791},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":733,"cache_hit":0,"new_prefill":26885,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":766,"cache_hit":0,"new_prefill":26885,"score_linear":121968.0,"score_lmetric":26885},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":26885,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281812.677479603,"t_first_token_unix":1779712677.6914227,"t_done":281813.937474022,"t_done_unix":1779712678.9514186},{"request_id":"1279412:12:1313518:319","session_id":"1279412","input_length":64755,"t_proxy_recv":281812.086936694,"t_decision_unix":1779712677.100878,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":64512,"affinity_cache_ratio":0.9962473940236275,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":64512,"estimated_new_tokens":243,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":64755,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":893,"cache_hit":0,"new_prefill":64755,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":64755,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":64755,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":64755,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26885,"ongoing_decode_tokens":0,"pending_prefill_tokens":26885,"num_requests":1,"active_p_offloads":0,"cached_blocks":733,"cache_hit":0,"new_prefill":64755,"score_linear":26885.0,"score_lmetric":91640},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":184808,"ongoing_decode_tokens":184808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":766,"cache_hit":0,"new_prefill":64755,"score_linear":184808.0,"score_lmetric":129510},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22686,"ongoing_decode_tokens":0,"pending_prefill_tokens":9886,"num_requests":1,"active_p_offloads":0,"cached_blocks":859,"cache_hit":64512,"new_prefill":243,"score_linear":-41826.0,"score_lmetric":10129}],"chosen_score_linear":-41826.0,"chosen_score_lmetric":10129,"t_first_token":281812.559487668,"t_first_token_unix":1779712677.573431,"t_done":281814.066926214,"t_done_unix":1779712679.0808697},{"request_id":"1269373:12:1312836:316","session_id":"1269373","input_length":62840,"t_proxy_recv":281809.717199606,"t_decision_unix":1779712674.7311409,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":59904,"affinity_cache_ratio":0.9532781667727562,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":59904,"estimated_new_tokens":2936,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13360,"ongoing_decode_tokens":0,"pending_prefill_tokens":13360,"num_requests":1,"active_p_offloads":0,"cached_blocks":649,"cache_hit":4096,"new_prefill":58744,"score_linear":9264.0,"score_lmetric":72104},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":44335,"ongoing_decode_tokens":44335,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":892,"cache_hit":0,"new_prefill":62840,"score_linear":44335.0,"score_lmetric":62840},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":62840,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":891,"cache_hit":7680,"new_prefill":55160,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82594,"ongoing_decode_tokens":37802,"pending_prefill_tokens":20216,"num_requests":3,"active_p_offloads":0,"cached_blocks":837,"cache_hit":0,"new_prefill":62840,"score_linear":82594.0,"score_lmetric":249168},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26885,"ongoing_decode_tokens":0,"pending_prefill_tokens":26885,"num_requests":1,"active_p_offloads":0,"cached_blocks":733,"cache_hit":0,"new_prefill":62840,"score_linear":26885.0,"score_lmetric":89725},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":766,"cache_hit":59904,"new_prefill":2936,"score_linear":62064.0,"score_lmetric":2936},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":7680,"new_prefill":55160,"score_linear":-7680.0,"score_lmetric":0}],"chosen_score_linear":62064.0,"chosen_score_lmetric":2936,"t_first_token":281810.991538564,"t_first_token_unix":1779712676.005482,"t_done":281816.939290836,"t_done_unix":1779712681.9532347},{"request_id":"1294611:10:1314495:322","session_id":"1294611","input_length":45362,"t_proxy_recv":281815.75511676,"t_decision_unix":1779712680.7690578,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":44032,"affinity_cache_ratio":0.970680305101186,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":44032,"estimated_new_tokens":1330,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":45362,"score_linear":57051.0,"score_lmetric":98829},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":893,"cache_hit":44032,"new_prefill":1330,"score_linear":-44032.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":45362,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":891,"cache_hit":12800,"new_prefill":32562,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":45362,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":785,"cache_hit":0,"new_prefill":45362,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":184808,"ongoing_decode_tokens":184808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":766,"cache_hit":0,"new_prefill":45362,"score_linear":184808.0,"score_lmetric":90724},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":878,"cache_hit":0,"new_prefill":45362,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-44032.0,"chosen_score_lmetric":0,"t_first_token":281816.114520435,"t_first_token_unix":1779712681.1284642,"t_done":281817.580292384,"t_done_unix":1779712682.5942357},{"request_id":"1313181:2:1314940:323","session_id":"1313181","input_length":24306,"t_proxy_recv":281817.155696496,"t_decision_unix":1779712682.1696377,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":22528,"affinity_cache_ratio":0.9268493376121122,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":22528,"estimated_new_tokens":1778,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":24306,"score_linear":57051.0,"score_lmetric":77773},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":45362,"ongoing_decode_tokens":45362,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":893,"cache_hit":0,"new_prefill":24306,"score_linear":45362.0,"score_lmetric":24306},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":24306,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":24306,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":24306,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":785,"cache_hit":0,"new_prefill":24306,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":24306,"score_linear":121968.0,"score_lmetric":24306},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":878,"cache_hit":22528,"new_prefill":1778,"score_linear":-22528.0,"score_lmetric":0}],"chosen_score_linear":-22528.0,"chosen_score_lmetric":0,"t_first_token":281817.498505006,"t_first_token_unix":1779712682.5124485,"t_done":281818.087163863,"t_done_unix":1779712683.1011071},{"request_id":"1315313:1:1315313:325","session_id":"1315313","input_length":7,"t_proxy_recv":281818.386585626,"t_decision_unix":1779712683.4005265,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":7,"score_linear":57051.0,"score_lmetric":53474},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":895,"cache_hit":0,"new_prefill":7,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":7,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":7,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":7,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":32440,"ongoing_decode_tokens":0,"pending_prefill_tokens":14520,"num_requests":1,"active_p_offloads":0,"cached_blocks":785,"cache_hit":0,"new_prefill":7,"score_linear":32440.0,"score_lmetric":14527},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":7,"score_linear":121968.0,"score_lmetric":7},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":881,"cache_hit":0,"new_prefill":7,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281818.402222666,"t_first_token_unix":1779712683.4161658,"t_done":281818.406753229,"t_done_unix":1779712683.420696},{"request_id":"1310590:2:1315212:324","session_id":"1310590","input_length":32440,"t_proxy_recv":281818.035019427,"t_decision_unix":1779712683.0489604,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":17920,"affinity_cache_ratio":0.5524044389642416,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":17920,"estimated_new_tokens":14520,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":32440,"score_linear":57051.0,"score_lmetric":85907},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":895,"cache_hit":0,"new_prefill":32440,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":32440,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":32440,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":32440,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":785,"cache_hit":17920,"new_prefill":14520,"score_linear":-17920.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":32440,"score_linear":121968.0,"score_lmetric":32440},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":24306,"ongoing_decode_tokens":24306,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":878,"cache_hit":0,"new_prefill":32440,"score_linear":24306.0,"score_lmetric":32440}],"chosen_score_linear":-17920.0,"chosen_score_lmetric":0,"t_first_token":281820.744037711,"t_first_token_unix":1779712685.7579818,"t_done":281820.894197524,"t_done_unix":1779712685.9081411},{"request_id":"1309040:2:1315488:326","session_id":"1309040","input_length":23097,"t_proxy_recv":281819.107159249,"t_decision_unix":1779712684.1211002,"policy":"unified","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":4608,"affinity_cache_ratio":0.1995064294064164,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":18489,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":23097,"score_linear":57051.0,"score_lmetric":76564},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":895,"cache_hit":0,"new_prefill":23097,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":23097,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":891,"cache_hit":4608,"new_prefill":18489,"score_linear":-4608.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":23097,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":32440,"ongoing_decode_tokens":0,"pending_prefill_tokens":14520,"num_requests":1,"active_p_offloads":0,"cached_blocks":785,"cache_hit":0,"new_prefill":23097,"score_linear":32440.0,"score_lmetric":37617},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":23097,"score_linear":121968.0,"score_lmetric":23097},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":881,"cache_hit":0,"new_prefill":23097,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":281821.452052632,"t_first_token_unix":1779712686.4659963,"t_done":281821.815377546,"t_done_unix":1779712686.829321},{"request_id":"1279412:13:1315780:328","session_id":"1279412","input_length":64949,"t_proxy_recv":281820.172781401,"t_decision_unix":1779712685.1867223,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":64512,"affinity_cache_ratio":0.9932716439052179,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":64512,"estimated_new_tokens":437,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":64949,"score_linear":57051.0,"score_lmetric":118416},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":895,"cache_hit":0,"new_prefill":64949,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":64949,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":23097,"ongoing_decode_tokens":0,"pending_prefill_tokens":18489,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":64949,"score_linear":23097.0,"score_lmetric":83438},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":64949,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":32440,"ongoing_decode_tokens":0,"pending_prefill_tokens":14520,"num_requests":1,"active_p_offloads":0,"cached_blocks":785,"cache_hit":0,"new_prefill":64949,"score_linear":32440.0,"score_lmetric":79469},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":64949,"score_linear":121968.0,"score_lmetric":64949},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":16960,"ongoing_decode_tokens":0,"pending_prefill_tokens":9280,"num_requests":1,"active_p_offloads":0,"cached_blocks":881,"cache_hit":64512,"new_prefill":437,"score_linear":-47552.0,"score_lmetric":9717}],"chosen_score_linear":-47552.0,"chosen_score_lmetric":9717,"t_first_token":281820.57389115,"t_first_token_unix":1779712685.587835,"t_done":281823.757120273,"t_done_unix":1779712688.771064},{"request_id":"1316848:1:1316848:331","session_id":"1316848","input_length":5985,"t_proxy_recv":281823.728215557,"t_decision_unix":1779712688.742157,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":1024,"estimated_new_tokens":4961,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":5985,"score_linear":57051.0,"score_lmetric":59452},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":24487,"ongoing_decode_tokens":0,"pending_prefill_tokens":24487,"num_requests":1,"active_p_offloads":0,"cached_blocks":895,"cache_hit":0,"new_prefill":5985,"score_linear":24487.0,"score_lmetric":30472},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":5985,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":927,"cache_hit":1024,"new_prefill":4961,"score_linear":-1024.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":5985,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":813,"cache_hit":0,"new_prefill":5985,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":5985,"score_linear":121968.0,"score_lmetric":5985},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":81909,"ongoing_decode_tokens":81909,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":881,"cache_hit":0,"new_prefill":5985,"score_linear":81909.0,"score_lmetric":11970}],"chosen_score_linear":-1024.0,"chosen_score_lmetric":0,"t_first_token":281824.069503627,"t_first_token_unix":1779712689.0834475,"t_done":281824.254758319,"t_done_unix":1779712689.2687016},{"request_id":"1316201:1:1316201:329","session_id":"1316201","input_length":24487,"t_proxy_recv":281821.639338473,"t_decision_unix":1779712686.6532798,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":24487,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":24487,"score_linear":57051.0,"score_lmetric":77954},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":895,"cache_hit":0,"new_prefill":24487,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":24487,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":23097,"ongoing_decode_tokens":23097,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":24487,"score_linear":23097.0,"score_lmetric":24487},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":24487,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":813,"cache_hit":0,"new_prefill":24487,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":24487,"score_linear":121968.0,"score_lmetric":24487},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":81909,"ongoing_decode_tokens":81909,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":881,"cache_hit":0,"new_prefill":24487,"score_linear":81909.0,"score_lmetric":48974}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281824.44218398,"t_first_token_unix":1779712689.4561276,"t_done":281824.734314769,"t_done_unix":1779712689.7482586},{"request_id":"1315567:1:1315567:327","session_id":"1315567","input_length":16960,"t_proxy_recv":281819.378037567,"t_decision_unix":1779712684.3919785,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":9280,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":4096,"new_prefill":12864,"score_linear":52955.0,"score_lmetric":66331},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":895,"cache_hit":0,"new_prefill":16960,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":16960,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":23097,"ongoing_decode_tokens":0,"pending_prefill_tokens":18489,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":7680,"new_prefill":9280,"score_linear":15417.0,"score_lmetric":27769},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":16960,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":32440,"ongoing_decode_tokens":0,"pending_prefill_tokens":14520,"num_requests":1,"active_p_offloads":0,"cached_blocks":785,"cache_hit":0,"new_prefill":16960,"score_linear":32440.0,"score_lmetric":31480},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":772,"cache_hit":7680,"new_prefill":9280,"score_linear":114288.0,"score_lmetric":9280},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":881,"cache_hit":7680,"new_prefill":9280,"score_linear":-7680.0,"score_lmetric":0}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":281820.488155153,"t_first_token_unix":1779712685.502099,"t_done":281825.066342907,"t_done_unix":1779712690.0802863},{"request_id":"1317160:1:1317160:332","session_id":"1317160","input_length":3441,"t_proxy_recv":281824.881229079,"t_decision_unix":1779712689.8951705,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3441,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":3441,"score_linear":57051.0,"score_lmetric":56908},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":3441,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":3441,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":3441,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":3441,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":813,"cache_hit":0,"new_prefill":3441,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":3441,"score_linear":121968.0,"score_lmetric":3441},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":16960,"ongoing_decode_tokens":16960,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":882,"cache_hit":0,"new_prefill":3441,"score_linear":16960.0,"score_lmetric":3441}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281825.098125197,"t_first_token_unix":1779712690.1120687,"t_done":281825.254628893,"t_done_unix":1779712690.2685723},{"request_id":"1316848:2:1317438:333","session_id":"1316848","input_length":6040,"t_proxy_recv":281825.962638828,"t_decision_unix":1779712690.9765801,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":5632,"affinity_cache_ratio":0.9324503311258279,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":5632,"estimated_new_tokens":408,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":6040,"score_linear":57051.0,"score_lmetric":59507},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":6040,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":6040,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":5632,"new_prefill":408,"score_linear":-5632.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":6040,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":819,"cache_hit":0,"new_prefill":6040,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":6040,"score_linear":121968.0,"score_lmetric":6040},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":6040,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-5632.0,"chosen_score_lmetric":0,"t_first_token":281825.990721691,"t_first_token_unix":1779712691.0046651,"t_done":281826.403667786,"t_done_unix":1779712691.4176114},{"request_id":"1317160:2:1317773:336","session_id":"1317160","input_length":7440,"t_proxy_recv":281827.28590577,"t_decision_unix":1779712692.299847,"policy":"unified","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":3072,"affinity_cache_ratio":0.4129032258064516,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":3072,"estimated_new_tokens":4368,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57051,"ongoing_decode_tokens":57051,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":7440,"score_linear":57051.0,"score_lmetric":7440},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":7440,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":7440,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":7440,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":7440,"score_linear":79663.0,"score_lmetric":86591},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":819,"cache_hit":3072,"new_prefill":4368,"score_linear":-3072.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":121968,"pending_prefill_tokens":3159,"num_requests":2,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":7440,"score_linear":188103.0,"score_lmetric":21198},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":7440,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-3072.0,"chosen_score_lmetric":0,"t_first_token":281827.625972786,"t_first_token_unix":1779712692.639916,"t_done":281827.750167413,"t_done_unix":1779712692.7641106},{"request_id":"1316201:2:1317776:337","session_id":"1316201","input_length":24768,"t_proxy_recv":281827.300290187,"t_decision_unix":1779712692.3142316,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9715762273901809,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":704,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57051,"ongoing_decode_tokens":57051,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":24768,"score_linear":57051.0,"score_lmetric":24768},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":942,"cache_hit":24064,"new_prefill":704,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":24768,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":24768,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":24768,"score_linear":79663.0,"score_lmetric":103919},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":7440,"ongoing_decode_tokens":0,"pending_prefill_tokens":4368,"num_requests":1,"active_p_offloads":0,"cached_blocks":819,"cache_hit":0,"new_prefill":24768,"score_linear":7440.0,"score_lmetric":29136},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":121968,"pending_prefill_tokens":3159,"num_requests":2,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":24768,"score_linear":188103.0,"score_lmetric":55854},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":24768,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":281827.425500325,"t_first_token_unix":1779712692.4394445,"t_done":281827.968323714,"t_done_unix":1779712692.9822667},{"request_id":"1318031:1:1318031:338","session_id":"1318031","input_length":5219,"t_proxy_recv":281827.943271737,"t_decision_unix":1779712692.9572132,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5219,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57051,"ongoing_decode_tokens":57051,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":5219,"score_linear":57051.0,"score_lmetric":5219},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":24768,"ongoing_decode_tokens":24768,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":5219,"score_linear":24768.0,"score_lmetric":5219},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":5219,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":5219,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":5219,"score_linear":79663.0,"score_lmetric":84370},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":827,"cache_hit":0,"new_prefill":5219,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":5219,"score_linear":188103.0,"score_lmetric":10438},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":5219,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281828.276539812,"t_first_token_unix":1779712693.290483,"t_done":281828.499108099,"t_done_unix":1779712693.5130515},{"request_id":"1318036:1:1318036:339","session_id":"1318036","input_length":2696,"t_proxy_recv":281827.947641539,"t_decision_unix":1779712692.961583,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":2696,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57051,"ongoing_decode_tokens":57051,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":2696,"score_linear":57051.0,"score_lmetric":2696},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":24768,"ongoing_decode_tokens":24768,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":2696,"score_linear":24768.0,"score_lmetric":2696},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":2696,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":2696,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":2696,"score_linear":79663.0,"score_lmetric":81847},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":5219,"ongoing_decode_tokens":0,"pending_prefill_tokens":5219,"num_requests":1,"active_p_offloads":0,"cached_blocks":827,"cache_hit":0,"new_prefill":2696,"score_linear":5219.0,"score_lmetric":7915},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":2696,"score_linear":188103.0,"score_lmetric":5392},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":2696,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281828.168987541,"t_first_token_unix":1779712693.182931,"t_done":281828.662485329,"t_done_unix":1779712693.6764283},{"request_id":"1314357:1:1314357:321","session_id":"1314357","input_length":57051,"t_proxy_recv":281815.282617266,"t_decision_unix":1779712680.2965586,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":3584,"estimated_new_tokens":53467,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":675,"cache_hit":3584,"new_prefill":53467,"score_linear":-3584.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":893,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":785,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":184808,"ongoing_decode_tokens":184808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":766,"cache_hit":0,"new_prefill":57051,"score_linear":184808.0,"score_lmetric":114102},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":878,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-3584.0,"chosen_score_lmetric":0,"t_first_token":281826.859784819,"t_first_token_unix":1779712691.8737285,"t_done":281828.768098366,"t_done_unix":1779712693.782042},{"request_id":"1318604:1:1318604:341","session_id":"1318604","input_length":14412,"t_proxy_recv":281829.991303702,"t_decision_unix":1779712695.0052445,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":14412,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":14412,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":14412,"score_linear":62618.0,"score_lmetric":18150},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":14412,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":14412,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":14412,"score_linear":79663.0,"score_lmetric":93563},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":837,"cache_hit":0,"new_prefill":14412,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":14412,"score_linear":188103.0,"score_lmetric":28824},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":14412,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281831.237394728,"t_first_token_unix":1779712696.2513385,"t_done":281831.459508541,"t_done_unix":1779712696.4734516},{"request_id":"1316848:3:1318975:343","session_id":"1316848","input_length":6298,"t_proxy_recv":281831.362818445,"t_decision_unix":1779712696.3767598,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":5632,"affinity_cache_ratio":0.8942521435376309,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":5632,"estimated_new_tokens":666,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":6298,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":6298,"score_linear":62618.0,"score_lmetric":10036},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":6298,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":14412,"ongoing_decode_tokens":14412,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":936,"cache_hit":5632,"new_prefill":666,"score_linear":8780.0,"score_lmetric":666},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":6298,"score_linear":79663.0,"score_lmetric":85449},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":837,"cache_hit":0,"new_prefill":6298,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":6298,"score_linear":188103.0,"score_lmetric":12596},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":6298,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":8780.0,"chosen_score_lmetric":666,"t_first_token":281831.422525797,"t_first_token_unix":1779712696.4364693,"t_done":281831.708337987,"t_done_unix":1779712696.7222817},{"request_id":"1269373:13:1317516:334","session_id":"1269373","input_length":66135,"t_proxy_recv":281826.225240573,"t_decision_unix":1779712691.2391818,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":62976,"affinity_cache_ratio":0.9522340666817872,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":62976,"estimated_new_tokens":3159,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":4096,"new_prefill":62039,"score_linear":52955.0,"score_lmetric":115506},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":66135,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":66135,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":6040,"ongoing_decode_tokens":6040,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":936,"cache_hit":7680,"new_prefill":58455,"score_linear":-1640.0,"score_lmetric":58455},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":66135,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":819,"cache_hit":0,"new_prefill":66135,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":772,"cache_hit":62976,"new_prefill":3159,"score_linear":58992.0,"score_lmetric":3159},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":7680,"new_prefill":58455,"score_linear":-7680.0,"score_lmetric":0}],"chosen_score_linear":58992.0,"chosen_score_lmetric":3159,"t_first_token":281827.562601782,"t_first_token_unix":1779712692.5765529,"t_done":281835.025464606,"t_done_unix":1779712700.0394077},{"request_id":"1318604:2:1319895:347","session_id":"1318604","input_length":15745,"t_proxy_recv":281834.50019537,"t_decision_unix":1779712699.5141366,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":14336,"affinity_cache_ratio":0.9105112734201334,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":1409,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":15745,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":15745,"score_linear":62618.0,"score_lmetric":15745},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":15745,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21140,"ongoing_decode_tokens":0,"pending_prefill_tokens":14996,"num_requests":1,"active_p_offloads":0,"cached_blocks":965,"cache_hit":14336,"new_prefill":1409,"score_linear":6804.0,"score_lmetric":16405},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":15745,"score_linear":79663.0,"score_lmetric":94896},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":837,"cache_hit":0,"new_prefill":15745,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":15745,"score_linear":188103.0,"score_lmetric":31490},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":71854,"ongoing_decode_tokens":0,"pending_prefill_tokens":6830,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":15745,"score_linear":71854.0,"score_lmetric":22575}],"chosen_score_linear":6804.0,"chosen_score_lmetric":16405,"t_first_token":281836.338394949,"t_first_token_unix":1779712701.3523383,"t_done":281836.590509854,"t_done_unix":1779712701.6044533},{"request_id":"1279412:14:1319059:344","session_id":"1279412","input_length":71854,"t_proxy_recv":281831.722438962,"t_decision_unix":1779712696.7363799,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9049461407854816,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":65024,"estimated_new_tokens":6830,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":71854,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":71854,"score_linear":62618.0,"score_lmetric":75592},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":71854,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":965,"cache_hit":0,"new_prefill":71854,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":71854,"score_linear":79663.0,"score_lmetric":151005},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":837,"cache_hit":0,"new_prefill":71854,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":71854,"score_linear":188103.0,"score_lmetric":143708},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":65024,"new_prefill":6830,"score_linear":-65024.0,"score_lmetric":0}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":281834.682529748,"t_first_token_unix":1779712699.6964731,"t_done":281837.571649946,"t_done_unix":1779712702.5855935},{"request_id":"1320630:1:1320630:348","session_id":"1320630","input_length":3099,"t_proxy_recv":281837.282202056,"t_decision_unix":1779712702.2961433,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3099,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":50450,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":3099,"score_linear":54546.0,"score_lmetric":53549},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":3099,"score_linear":62618.0,"score_lmetric":3099},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":3099,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21140,"ongoing_decode_tokens":21140,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":967,"cache_hit":0,"new_prefill":3099,"score_linear":21140.0,"score_lmetric":3099},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":3099,"score_linear":79663.0,"score_lmetric":82250},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":837,"cache_hit":0,"new_prefill":3099,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":3099,"score_linear":121968.0,"score_lmetric":3099},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":71854,"ongoing_decode_tokens":71854,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":3099,"score_linear":71854.0,"score_lmetric":3099}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281837.479572814,"t_first_token_unix":1779712702.493516,"t_done":281837.677503663,"t_done_unix":1779712702.6914465},{"request_id":"1316848:4:1319820:345","session_id":"1316848","input_length":21140,"t_proxy_recv":281834.270640987,"t_decision_unix":1779712699.2845821,"policy":"unified","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":6144,"affinity_cache_ratio":0.29063386944181646,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":6144,"estimated_new_tokens":14996,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":21140,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":21140,"score_linear":62618.0,"score_lmetric":21140},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":21140,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":965,"cache_hit":6144,"new_prefill":14996,"score_linear":-6144.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":21140,"score_linear":79663.0,"score_lmetric":100291},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":837,"cache_hit":0,"new_prefill":21140,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":21140,"score_linear":188103.0,"score_lmetric":42280},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":71854,"ongoing_decode_tokens":0,"pending_prefill_tokens":6830,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":21140,"score_linear":71854.0,"score_lmetric":27970}],"chosen_score_linear":-6144.0,"chosen_score_lmetric":0,"t_first_token":281836.179248508,"t_first_token_unix":1779712701.193192,"t_done":281838.580997376,"t_done_unix":1779712703.5949404},{"request_id":"1318604:3:1320865:350","session_id":"1318604","input_length":17883,"t_proxy_recv":281838.126520799,"t_decision_unix":1779712703.1404622,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":15360,"affinity_cache_ratio":0.8589162892132193,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":15360,"estimated_new_tokens":2523,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":50450,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":17883,"score_linear":54546.0,"score_lmetric":68333},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":17883,"score_linear":62618.0,"score_lmetric":17883},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":17883,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21140,"ongoing_decode_tokens":21140,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":967,"cache_hit":15360,"new_prefill":2523,"score_linear":5780.0,"score_lmetric":2523},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":17883,"score_linear":79663.0,"score_lmetric":97034},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":843,"cache_hit":0,"new_prefill":17883,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":17883,"score_linear":121968.0,"score_lmetric":17883},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":914,"cache_hit":0,"new_prefill":17883,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":5780.0,"chosen_score_lmetric":2523,"t_first_token":281838.471728341,"t_first_token_unix":1779712703.4856718,"t_done":281838.728980568,"t_done_unix":1779712703.742924},{"request_id":"1318036:2:1321282:353","session_id":"1318036","input_length":2677,"t_proxy_recv":281839.586342377,"t_decision_unix":1779712704.6002836,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":2560,"affinity_cache_ratio":0.9562943593574897,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":2560,"estimated_new_tokens":117,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":50450,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":2677,"score_linear":54546.0,"score_lmetric":53127},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":2677,"score_linear":62618.0,"score_lmetric":2677},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":2560,"new_prefill":117,"score_linear":-2560.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":33906,"ongoing_decode_tokens":0,"pending_prefill_tokens":2674,"num_requests":1,"active_p_offloads":0,"cached_blocks":1001,"cache_hit":0,"new_prefill":2677,"score_linear":33906.0,"score_lmetric":5351},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":2677,"score_linear":79663.0,"score_lmetric":81828},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":8095,"ongoing_decode_tokens":8095,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":843,"cache_hit":0,"new_prefill":2677,"score_linear":8095.0,"score_lmetric":2677},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":2677,"score_linear":121968.0,"score_lmetric":2677},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":914,"cache_hit":0,"new_prefill":2677,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-2560.0,"chosen_score_lmetric":0,"t_first_token":281839.607975391,"t_first_token_unix":1779712704.6219182,"t_done":281839.615850714,"t_done_unix":1779712704.6297936},{"request_id":"1321042:1:1321042:351","session_id":"1321042","input_length":8095,"t_proxy_recv":281838.776632152,"t_decision_unix":1779712703.7905731,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8095,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":50450,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":8095,"score_linear":54546.0,"score_lmetric":58545},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":8095,"score_linear":62618.0,"score_lmetric":8095},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":8095,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1001,"cache_hit":0,"new_prefill":8095,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":8095,"score_linear":79663.0,"score_lmetric":87246},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":843,"cache_hit":0,"new_prefill":8095,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":8095,"score_linear":121968.0,"score_lmetric":8095},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":914,"cache_hit":0,"new_prefill":8095,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281839.346784836,"t_first_token_unix":1779712704.3607283,"t_done":281839.659057079,"t_done_unix":1779712704.673},{"request_id":"1277428:4:1321259:352","session_id":"1277428","input_length":33906,"t_proxy_recv":281839.533951752,"t_decision_unix":1779712704.547893,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9211349023771603,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":31232,"estimated_new_tokens":2674,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":50450,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":33906,"score_linear":54546.0,"score_lmetric":84356},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":4608,"new_prefill":29298,"score_linear":58010.0,"score_lmetric":29298},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":33906,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1001,"cache_hit":31232,"new_prefill":2674,"score_linear":-31232.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":33906,"score_linear":79663.0,"score_lmetric":113057},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":8095,"ongoing_decode_tokens":8095,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":843,"cache_hit":0,"new_prefill":33906,"score_linear":8095.0,"score_lmetric":33906},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":33906,"score_linear":121968.0,"score_lmetric":33906},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":914,"cache_hit":0,"new_prefill":33906,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-31232.0,"chosen_score_lmetric":0,"t_first_token":281840.143332418,"t_first_token_unix":1779712705.157276,"t_done":281840.787525003,"t_done_unix":1779712705.8014681},{"request_id":"1318604:4:1321889:356","session_id":"1318604","input_length":21033,"t_proxy_recv":281841.717620314,"t_decision_unix":1779712706.7315614,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":17920,"affinity_cache_ratio":0.8519944848571293,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":17920,"estimated_new_tokens":3113,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":50450,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":21033,"score_linear":54546.0,"score_lmetric":71483},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":21033,"score_linear":62618.0,"score_lmetric":21033},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":21033,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1006,"cache_hit":17920,"new_prefill":3113,"score_linear":-17920.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":21033,"score_linear":79663.0,"score_lmetric":100184},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":858,"cache_hit":0,"new_prefill":21033,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188808,"ongoing_decode_tokens":188808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":21033,"score_linear":188808.0,"score_lmetric":42066},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":72402,"ongoing_decode_tokens":72402,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":914,"cache_hit":0,"new_prefill":21033,"score_linear":72402.0,"score_lmetric":21033}],"chosen_score_linear":-17920.0,"chosen_score_lmetric":0,"t_first_token":281842.236951785,"t_first_token_unix":1779712707.2508953,"t_done":281842.475913463,"t_done_unix":1779712707.4898567},{"request_id":"1279412:15:1321613:355","session_id":"1279412","input_length":72402,"t_proxy_recv":281840.712975463,"t_decision_unix":1779712705.726916,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":71680,"affinity_cache_ratio":0.990027899781774,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71680,"estimated_new_tokens":722,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":50450,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":72402,"score_linear":54546.0,"score_lmetric":122852},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":72402,"score_linear":62618.0,"score_lmetric":72402},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":72402,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":33906,"ongoing_decode_tokens":33906,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1001,"cache_hit":0,"new_prefill":72402,"score_linear":33906.0,"score_lmetric":72402},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":72402,"score_linear":79663.0,"score_lmetric":151553},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":858,"cache_hit":0,"new_prefill":72402,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188808,"ongoing_decode_tokens":188808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":72402,"score_linear":188808.0,"score_lmetric":144804},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":914,"cache_hit":71680,"new_prefill":722,"score_linear":-71680.0,"score_lmetric":0}],"chosen_score_linear":-71680.0,"chosen_score_lmetric":0,"t_first_token":281840.954481809,"t_first_token_unix":1779712705.9684253,"t_done":281843.202338786,"t_done_unix":1779712708.2162821},{"request_id":"1316848:5:1322192:357","session_id":"1316848","input_length":21471,"t_proxy_recv":281842.895810952,"t_decision_unix":1779712707.9097524,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":20992,"affinity_cache_ratio":0.9776908388058311,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":20992,"estimated_new_tokens":479,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":50450,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":21471,"score_linear":54546.0,"score_lmetric":71921},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":21471,"score_linear":62618.0,"score_lmetric":21471},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":21471,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1012,"cache_hit":20992,"new_prefill":479,"score_linear":-20992.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":21471,"score_linear":79663.0,"score_lmetric":100622},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":858,"cache_hit":0,"new_prefill":21471,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188808,"ongoing_decode_tokens":188808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":21471,"score_linear":188808.0,"score_lmetric":42942},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":72402,"ongoing_decode_tokens":72402,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":914,"cache_hit":0,"new_prefill":21471,"score_linear":72402.0,"score_lmetric":21471}],"chosen_score_linear":-20992.0,"chosen_score_lmetric":0,"t_first_token":281842.94546404,"t_first_token_unix":1779712707.959407,"t_done":281843.483234731,"t_done_unix":1779712708.4971802},{"request_id":"1269373:14:1321512:354","session_id":"1269373","input_length":66840,"t_proxy_recv":281840.373261326,"t_decision_unix":1779712705.387202,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":66560,"affinity_cache_ratio":0.9958108916816277,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":66560,"estimated_new_tokens":280,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":50450,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":4096,"new_prefill":62744,"score_linear":50450.0,"score_lmetric":113194},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":66840,"score_linear":62618.0,"score_lmetric":66840},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":66840,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":33906,"ongoing_decode_tokens":33906,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1001,"cache_hit":7680,"new_prefill":59160,"score_linear":26226.0,"score_lmetric":59160},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":66840,"score_linear":79663.0,"score_lmetric":145991},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":858,"cache_hit":0,"new_prefill":66840,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":66560,"new_prefill":280,"score_linear":55408.0,"score_lmetric":280},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":914,"cache_hit":7680,"new_prefill":59160,"score_linear":-7680.0,"score_lmetric":0}],"chosen_score_linear":55408.0,"chosen_score_lmetric":280,"t_first_token":281840.599433299,"t_first_token_unix":1779712705.6133769,"t_done":281844.655892105,"t_done_unix":1779712709.669836},{"request_id":"1322540:1:1322540:359","session_id":"1322540","input_length":8939,"t_proxy_recv":281844.178193103,"t_decision_unix":1779712709.192134,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8939,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":50450,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":8939,"score_linear":54546.0,"score_lmetric":59389},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":8939,"score_linear":62618.0,"score_lmetric":8939},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":8939,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1013,"cache_hit":0,"new_prefill":8939,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":8939,"score_linear":79663.0,"score_lmetric":88090},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":0,"pending_prefill_tokens":33182,"num_requests":1,"active_p_offloads":0,"cached_blocks":858,"cache_hit":0,"new_prefill":8939,"score_linear":33182.0,"score_lmetric":42121},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188808,"ongoing_decode_tokens":188808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":8939,"score_linear":188808.0,"score_lmetric":17878},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":915,"cache_hit":0,"new_prefill":8939,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281844.8939324,"t_first_token_unix":1779712709.9078765,"t_done":281845.155977527,"t_done_unix":1779712710.1699207},{"request_id":"1318031:2:1322552:360","session_id":"1318031","input_length":11145,"t_proxy_recv":281844.231800596,"t_decision_unix":1779712709.2457416,"policy":"unified","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":3,"affinity_cache_hit":5120,"affinity_cache_ratio":0.45939883355764916,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":11145,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":50450,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":11145,"score_linear":54546.0,"score_lmetric":61595},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":11145,"score_linear":62618.0,"score_lmetric":11145},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":11145,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1013,"cache_hit":0,"new_prefill":11145,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":11145,"score_linear":79663.0,"score_lmetric":90296},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":0,"pending_prefill_tokens":33182,"num_requests":1,"active_p_offloads":0,"cached_blocks":858,"cache_hit":5120,"new_prefill":6025,"score_linear":28062.0,"score_lmetric":39207},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188808,"ongoing_decode_tokens":188808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":11145,"score_linear":188808.0,"score_lmetric":22290},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":8939,"ongoing_decode_tokens":0,"pending_prefill_tokens":8939,"num_requests":1,"active_p_offloads":0,"cached_blocks":915,"cache_hit":0,"new_prefill":11145,"score_linear":8939.0,"score_lmetric":20084}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281845.115151442,"t_first_token_unix":1779712710.129095,"t_done":281845.167250724,"t_done_unix":1779712710.1811943},{"request_id":"1322540:2:1323519:362","session_id":"1322540","input_length":9016,"t_proxy_recv":281847.474902795,"t_decision_unix":1779712712.4888444,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":8704,"affinity_cache_ratio":0.9653948535936113,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":8704,"estimated_new_tokens":312,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":54546,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":9016,"score_linear":54546.0,"score_lmetric":9016},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":9016,"score_linear":62618.0,"score_lmetric":9016},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":9016,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":9016,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":9016,"score_linear":79663.0,"score_lmetric":88167},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":46190,"ongoing_decode_tokens":0,"pending_prefill_tokens":38510,"num_requests":2,"active_p_offloads":0,"cached_blocks":858,"cache_hit":0,"new_prefill":9016,"score_linear":46190.0,"score_lmetric":95052},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":9016,"score_linear":121968.0,"score_lmetric":9016},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":8704,"new_prefill":312,"score_linear":-8704.0,"score_lmetric":0}],"chosen_score_linear":-8704.0,"chosen_score_lmetric":0,"t_first_token":281847.508243033,"t_first_token_unix":1779712712.5221863,"t_done":281847.633836742,"t_done_unix":1779712712.6477797},{"request_id":"1315567:2:1319894:346","session_id":"1315567","input_length":54546,"t_proxy_recv":281834.505643024,"t_decision_unix":1779712699.5195844,"policy":"unified","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":0,"affinity_cache_hit":17408,"affinity_cache_ratio":0.31914347523191433,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":4096,"estimated_new_tokens":50450,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":779,"cache_hit":4096,"new_prefill":50450,"score_linear":-4096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":54546,"score_linear":62618.0,"score_lmetric":54546},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":54546,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":36885,"ongoing_decode_tokens":0,"pending_prefill_tokens":16405,"num_requests":2,"active_p_offloads":0,"cached_blocks":965,"cache_hit":7680,"new_prefill":46866,"score_linear":29205.0,"score_lmetric":126542},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":54546,"score_linear":79663.0,"score_lmetric":133697},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":837,"cache_hit":0,"new_prefill":54546,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":772,"cache_hit":7680,"new_prefill":46866,"score_linear":180423.0,"score_lmetric":93732},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":71854,"ongoing_decode_tokens":0,"pending_prefill_tokens":6830,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":17408,"new_prefill":37138,"score_linear":54446.0,"score_lmetric":43968}],"chosen_score_linear":-4096.0,"chosen_score_lmetric":0,"t_first_token":281845.414445778,"t_first_token_unix":1779712710.4283893,"t_done":281848.216862385,"t_done_unix":1779712713.2308059},{"request_id":"1316848:6:1323535:363","session_id":"1316848","input_length":24395,"t_proxy_recv":281847.526811115,"t_decision_unix":1779712712.5407526,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":21504,"affinity_cache_ratio":0.8814921090387374,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":21504,"estimated_new_tokens":2891,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":54546,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":24395,"score_linear":54546.0,"score_lmetric":24395},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":24395,"score_linear":62618.0,"score_lmetric":24395},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":24395,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":21504,"new_prefill":2891,"score_linear":-21504.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":24395,"score_linear":79663.0,"score_lmetric":103546},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":46190,"ongoing_decode_tokens":0,"pending_prefill_tokens":38510,"num_requests":2,"active_p_offloads":0,"cached_blocks":858,"cache_hit":0,"new_prefill":24395,"score_linear":46190.0,"score_lmetric":125810},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":24395,"score_linear":121968.0,"score_lmetric":24395},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":9016,"ongoing_decode_tokens":9016,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":24395,"score_linear":9016.0,"score_lmetric":24395}],"chosen_score_linear":-21504.0,"chosen_score_lmetric":0,"t_first_token":281848.094512627,"t_first_token_unix":1779712713.1084564,"t_done":281848.692443437,"t_done_unix":1779712713.706387},{"request_id":"1321042:2:1323492:361","session_id":"1321042","input_length":13008,"t_proxy_recv":281847.404817335,"t_decision_unix":1779712712.4187586,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":7680,"affinity_cache_ratio":0.5904059040590406,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":5328,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":54546,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":13008,"score_linear":54546.0,"score_lmetric":13008},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":13008,"score_linear":62618.0,"score_lmetric":13008},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":13008,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":13008,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":13008,"score_linear":79663.0,"score_lmetric":92159},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":0,"pending_prefill_tokens":33182,"num_requests":1,"active_p_offloads":0,"cached_blocks":858,"cache_hit":7680,"new_prefill":5328,"score_linear":25502.0,"score_lmetric":38510},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":13008,"score_linear":121968.0,"score_lmetric":13008},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":13008,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":25502.0,"chosen_score_lmetric":38510,"t_first_token":281848.197876871,"t_first_token_unix":1779712713.2118201,"t_done":281848.75455335,"t_done_unix":1779712713.7684968},{"request_id":"1320630:2:1323748:364","session_id":"1320630","input_length":11298,"t_proxy_recv":281848.246147196,"t_decision_unix":1779712713.260089,"policy":"unified","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":2,"affinity_cache_hit":3072,"affinity_cache_ratio":0.27190653212958044,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":11298,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":878,"cache_hit":0,"new_prefill":11298,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":11298,"score_linear":62618.0,"score_lmetric":11298},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":11298,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":24395,"ongoing_decode_tokens":24395,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":11298,"score_linear":24395.0,"score_lmetric":11298},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":11298,"score_linear":79663.0,"score_lmetric":11298},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":46190,"ongoing_decode_tokens":46190,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":858,"cache_hit":3072,"new_prefill":8226,"score_linear":43118.0,"score_lmetric":16452},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":11298,"score_linear":121968.0,"score_lmetric":11298},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":11298,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281849.147641744,"t_first_token_unix":1779712714.1615853,"t_done":281849.42930854,"t_done_unix":1779712714.4432526},{"request_id":"1324227:1:1324227:367","session_id":"1324227","input_length":14863,"t_proxy_recv":281850.042838155,"t_decision_unix":1779712715.0567796,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":14863,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":878,"cache_hit":0,"new_prefill":14863,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":14863,"score_linear":62618.0,"score_lmetric":14863},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":14863,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":14863,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":14863,"score_linear":79663.0,"score_lmetric":14863},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":14863,"score_linear":33182.0,"score_lmetric":14863},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":14863,"score_linear":121968.0,"score_lmetric":14863},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75624,"ongoing_decode_tokens":75624,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":14863,"score_linear":75624.0,"score_lmetric":14863}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281851.356168847,"t_first_token_unix":1779712716.3701122,"t_done":281851.60162967,"t_done_unix":1779712716.615573},{"request_id":"1279412:16:1323780:365","session_id":"1279412","input_length":75624,"t_proxy_recv":281848.349185809,"t_decision_unix":1779712713.3631272,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":72192,"affinity_cache_ratio":0.9546175817200888,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":72192,"estimated_new_tokens":3432,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":878,"cache_hit":0,"new_prefill":75624,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":75624,"score_linear":62618.0,"score_lmetric":75624},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":11298,"num_requests":1,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":75624,"score_linear":11298.0,"score_lmetric":86922},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":24395,"ongoing_decode_tokens":24395,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":75624,"score_linear":24395.0,"score_lmetric":75624},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":75624,"score_linear":79663.0,"score_lmetric":75624},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":46190,"ongoing_decode_tokens":46190,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":858,"cache_hit":0,"new_prefill":75624,"score_linear":46190.0,"score_lmetric":151248},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":75624,"score_linear":121968.0,"score_lmetric":75624},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":72192,"new_prefill":3432,"score_linear":-72192.0,"score_lmetric":0}],"chosen_score_linear":-72192.0,"chosen_score_lmetric":0,"t_first_token":281849.856613544,"t_first_token_unix":1779712714.870557,"t_done":281852.377996717,"t_done_unix":1779712717.39194},{"request_id":"1316848:7:1325031:371","session_id":"1316848","input_length":26994,"t_proxy_recv":281852.69932013,"t_decision_unix":1779712717.7132616,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":24064,"affinity_cache_ratio":0.8914573608950137,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":2930,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":4312,"ongoing_decode_tokens":0,"pending_prefill_tokens":4312,"num_requests":1,"active_p_offloads":0,"cached_blocks":878,"cache_hit":0,"new_prefill":26994,"score_linear":4312.0,"score_lmetric":31306},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":26994,"score_linear":62618.0,"score_lmetric":26994},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":26994,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1068,"cache_hit":24064,"new_prefill":2930,"score_linear":-24064.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":26994,"score_linear":79663.0,"score_lmetric":26994},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":26994,"score_linear":33182.0,"score_lmetric":26994},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":26994,"score_linear":121968.0,"score_lmetric":26994},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":939,"cache_hit":0,"new_prefill":26994,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":281853.261152175,"t_first_token_unix":1779712718.2750955,"t_done":281853.901866553,"t_done_unix":1779712718.9158096},{"request_id":"1322540:3:1325410:373","session_id":"1322540","input_length":12118,"t_proxy_recv":281854.070220817,"t_decision_unix":1779712719.0841622,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":8704,"affinity_cache_ratio":0.7182703416405347,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":8704,"estimated_new_tokens":3414,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":59442,"ongoing_decode_tokens":59442,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":878,"cache_hit":0,"new_prefill":12118,"score_linear":59442.0,"score_lmetric":24236},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":12118,"score_linear":62618.0,"score_lmetric":12118},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":12118,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":12118,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":12118,"score_linear":79663.0,"score_lmetric":12118},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":12118,"score_linear":33182.0,"score_lmetric":12118},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":12118,"score_linear":121968.0,"score_lmetric":12118},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":939,"cache_hit":8704,"new_prefill":3414,"score_linear":-8704.0,"score_lmetric":0}],"chosen_score_linear":-8704.0,"chosen_score_lmetric":0,"t_first_token":281854.428428399,"t_first_token_unix":1779712719.4423723,"t_done":281854.554833474,"t_done_unix":1779712719.5687768},{"request_id":"1324964:1:1324964:370","session_id":"1324964","input_length":4312,"t_proxy_recv":281852.456690589,"t_decision_unix":1779712717.4706318,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4312,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":878,"cache_hit":0,"new_prefill":4312,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":4312,"score_linear":62618.0,"score_lmetric":4312},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":4312,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1068,"cache_hit":0,"new_prefill":4312,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":4312,"score_linear":79663.0,"score_lmetric":4312},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":4312,"score_linear":33182.0,"score_lmetric":4312},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":4312,"score_linear":121968.0,"score_lmetric":4312},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":939,"cache_hit":0,"new_prefill":4312,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281852.727662332,"t_first_token_unix":1779712717.7416055,"t_done":281854.831907591,"t_done_unix":1779712719.8458507},{"request_id":"1317598:1:1317598:335","session_id":"1317598","input_length":79663,"t_proxy_recv":281826.530369943,"t_decision_unix":1779712691.5443106,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":512,"estimated_new_tokens":79151,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":79663,"score_linear":57051.0,"score_lmetric":133130},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":79663,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":79663,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":79663,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":512,"new_prefill":79151,"score_linear":-512.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":819,"cache_hit":0,"new_prefill":79663,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":121968,"pending_prefill_tokens":3159,"num_requests":2,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":79663,"score_linear":188103.0,"score_lmetric":165644},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":79663,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-512.0,"chosen_score_lmetric":0,"t_first_token":281848.206735471,"t_first_token_unix":1779712713.2206788,"t_done":281855.442536656,"t_done_unix":1779712720.4564805},{"request_id":"1325987:1:1325987:376","session_id":"1325987","input_length":1823,"t_proxy_recv":281856.164731509,"t_decision_unix":1779712721.1786726,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1823,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":887,"cache_hit":0,"new_prefill":1823,"score_linear":55130.0,"score_lmetric":1823},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":1823,"score_linear":62618.0,"score_lmetric":1823},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":1823,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":66703,"ongoing_decode_tokens":0,"pending_prefill_tokens":66703,"num_requests":1,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":1823,"score_linear":66703.0,"score_lmetric":68526},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81220,"ongoing_decode_tokens":81220,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1057,"cache_hit":0,"new_prefill":1823,"score_linear":81220.0,"score_lmetric":1823},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":1823,"score_linear":33182.0,"score_lmetric":1823},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":1823,"score_linear":191009.0,"score_lmetric":3646},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":945,"cache_hit":0,"new_prefill":1823,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281856.316277651,"t_first_token_unix":1779712721.330221,"t_done":281856.501453224,"t_done_unix":1779712721.5153964},{"request_id":"1326192:1:1326192:378","session_id":"1326192","input_length":9513,"t_proxy_recv":281857.058110696,"t_decision_unix":1779712722.0720522,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":9513,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":887,"cache_hit":0,"new_prefill":9513,"score_linear":55130.0,"score_lmetric":9513},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":9513,"score_linear":62618.0,"score_lmetric":9513},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":9513,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":66703,"ongoing_decode_tokens":0,"pending_prefill_tokens":66703,"num_requests":1,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":9513,"score_linear":66703.0,"score_lmetric":76216},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81220,"ongoing_decode_tokens":81220,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1057,"cache_hit":0,"new_prefill":9513,"score_linear":81220.0,"score_lmetric":9513},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":9513,"score_linear":33182.0,"score_lmetric":9513},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":9513,"score_linear":191009.0,"score_lmetric":19026},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":81008,"ongoing_decode_tokens":0,"pending_prefill_tokens":5232,"num_requests":1,"active_p_offloads":0,"cached_blocks":945,"cache_hit":0,"new_prefill":9513,"score_linear":81008.0,"score_lmetric":14745}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281857.840319785,"t_first_token_unix":1779712722.854263,"t_done":281858.383756672,"t_done_unix":1779712723.3976996},{"request_id":"1317598:2:1324364:368","session_id":"1317598","input_length":81220,"t_proxy_recv":281855.457840219,"t_decision_unix":1779712720.4717817,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":80384,"affinity_cache_ratio":0.9897069687269145,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":80384,"estimated_new_tokens":836,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":887,"cache_hit":0,"new_prefill":81220,"score_linear":55130.0,"score_lmetric":81220},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":81220,"score_linear":62618.0,"score_lmetric":81220},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":81220,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":66703,"ongoing_decode_tokens":0,"pending_prefill_tokens":66703,"num_requests":1,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":81220,"score_linear":66703.0,"score_lmetric":147923},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1057,"cache_hit":80384,"new_prefill":836,"score_linear":-80384.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":81220,"score_linear":33182.0,"score_lmetric":81220},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":121968,"pending_prefill_tokens":1969,"num_requests":2,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":81220,"score_linear":191009.0,"score_lmetric":166378},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":945,"cache_hit":0,"new_prefill":81220,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-80384.0,"chosen_score_lmetric":0,"t_first_token":281856.046190618,"t_first_token_unix":1779712721.0601344,"t_done":281858.897848059,"t_done_unix":1779712723.9117913},{"request_id":"1322540:4:1326861:380","session_id":"1322540","input_length":13249,"t_proxy_recv":281859.321974167,"t_decision_unix":1779712724.3359127,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":11776,"affinity_cache_ratio":0.8888217978715375,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":1473,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":887,"cache_hit":0,"new_prefill":13249,"score_linear":55130.0,"score_lmetric":13249},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":13249,"score_linear":62618.0,"score_lmetric":13249},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1072,"cache_hit":0,"new_prefill":13249,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":94823,"ongoing_decode_tokens":0,"pending_prefill_tokens":68199,"num_requests":2,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":13249,"score_linear":94823.0,"score_lmetric":162896},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":13249,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":13249,"score_linear":33182.0,"score_lmetric":13249},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":13249,"score_linear":191009.0,"score_lmetric":26498},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":81008,"ongoing_decode_tokens":0,"pending_prefill_tokens":5232,"num_requests":1,"active_p_offloads":0,"cached_blocks":945,"cache_hit":11776,"new_prefill":1473,"score_linear":69232.0,"score_lmetric":6705}],"chosen_score_linear":69232.0,"chosen_score_lmetric":6705,"t_first_token":281859.673616837,"t_first_token_unix":1779712724.6875606,"t_done":281859.931097801,"t_done_unix":1779712724.9450412},{"request_id":"1326192:2:1327234:381","session_id":"1326192","input_length":15253,"t_proxy_recv":281860.604730843,"t_decision_unix":1779712725.618672,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":9216,"affinity_cache_ratio":0.6042090080639874,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9216,"estimated_new_tokens":6037,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":887,"cache_hit":0,"new_prefill":15253,"score_linear":55130.0,"score_lmetric":15253},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":15253,"score_linear":62618.0,"score_lmetric":15253},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1072,"cache_hit":9216,"new_prefill":6037,"score_linear":-9216.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":94823,"ongoing_decode_tokens":0,"pending_prefill_tokens":68199,"num_requests":2,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":15253,"score_linear":94823.0,"score_lmetric":166904},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":15253,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":15253,"score_linear":33182.0,"score_lmetric":15253},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":15253,"score_linear":191009.0,"score_lmetric":30506},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":15253,"score_linear":81008.0,"score_lmetric":15253}],"chosen_score_linear":-9216.0,"chosen_score_lmetric":0,"t_first_token":281861.300831691,"t_first_token_unix":1779712726.3147757,"t_done":281861.682117196,"t_done_unix":1779712726.6960604},{"request_id":"1324964:2:1327382:382","session_id":"1324964","input_length":5146,"t_proxy_recv":281861.207917373,"t_decision_unix":1779712726.2218585,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":4608,"affinity_cache_ratio":0.895452778857365,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":538,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":887,"cache_hit":4608,"new_prefill":538,"score_linear":50522.0,"score_lmetric":538},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":5146,"score_linear":62618.0,"score_lmetric":5146},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":15253,"ongoing_decode_tokens":0,"pending_prefill_tokens":6037,"num_requests":1,"active_p_offloads":0,"cached_blocks":1072,"cache_hit":0,"new_prefill":5146,"score_linear":15253.0,"score_lmetric":11183},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":94823,"ongoing_decode_tokens":0,"pending_prefill_tokens":68199,"num_requests":2,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":5146,"score_linear":94823.0,"score_lmetric":146690},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":5146,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":5146,"score_linear":33182.0,"score_lmetric":5146},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":5146,"score_linear":191009.0,"score_lmetric":10292},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":5146,"score_linear":81008.0,"score_lmetric":5146}],"chosen_score_linear":50522.0,"chosen_score_lmetric":538,"t_first_token":281861.373909329,"t_first_token_unix":1779712726.3878522,"t_done":281862.19766486,"t_done_unix":1779712727.2116082},{"request_id":"1266668:12:1303535:276","session_id":"1266668","input_length":121968,"t_proxy_recv":281776.874150159,"t_decision_unix":1779712641.888091,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":97792,"affinity_cache_ratio":0.8017840745113473,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":97792,"estimated_new_tokens":24176,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":112736,"ongoing_decode_tokens":112736,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":584,"cache_hit":4096,"new_prefill":117872,"score_linear":108640.0,"score_lmetric":353616},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":121968,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14944,"ongoing_decode_tokens":0,"pending_prefill_tokens":14944,"num_requests":1,"active_p_offloads":0,"cached_blocks":973,"cache_hit":0,"new_prefill":121968,"score_linear":14944.0,"score_lmetric":136912},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":79803,"ongoing_decode_tokens":0,"pending_prefill_tokens":443,"num_requests":2,"active_p_offloads":0,"cached_blocks":871,"cache_hit":7680,"new_prefill":114288,"score_linear":72123.0,"score_lmetric":229462},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":15081,"ongoing_decode_tokens":0,"pending_prefill_tokens":15081,"num_requests":1,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":121968,"score_linear":15081.0,"score_lmetric":137049},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":121968,"score_linear":34706.0,"score_lmetric":121968},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":54451,"ongoing_decode_tokens":54451,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":748,"cache_hit":97792,"new_prefill":24176,"score_linear":-43341.0,"score_lmetric":24176},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":688,"cache_hit":7680,"new_prefill":114288,"score_linear":-7680.0,"score_lmetric":0}],"chosen_score_linear":-43341.0,"chosen_score_lmetric":24176,"t_first_token":281792.847653866,"t_first_token_unix":1779712657.8615975,"t_done":281863.614897572,"t_done_unix":1779712728.6288416},{"request_id":"1269373:15:1325630:375","session_id":"1269373","input_length":69041,"t_proxy_recv":281854.915428556,"t_decision_unix":1779712719.92937,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":67072,"affinity_cache_ratio":0.971480714358135,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":67072,"estimated_new_tokens":1969,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":887,"cache_hit":7680,"new_prefill":61361,"score_linear":47450.0,"score_lmetric":61361},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":69041,"score_linear":62618.0,"score_lmetric":69041},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":69041,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":66703,"ongoing_decode_tokens":0,"pending_prefill_tokens":66703,"num_requests":1,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":7680,"new_prefill":61361,"score_linear":59023.0,"score_lmetric":128064},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":69041,"score_linear":79663.0,"score_lmetric":69041},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":69041,"score_linear":33182.0,"score_lmetric":69041},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":780,"cache_hit":67072,"new_prefill":1969,"score_linear":54896.0,"score_lmetric":1969},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":945,"cache_hit":7680,"new_prefill":61361,"score_linear":-7680.0,"score_lmetric":0}],"chosen_score_linear":54896.0,"chosen_score_lmetric":1969,"t_first_token":281855.922442516,"t_first_token_unix":1779712720.9363866,"t_done":281864.257297994,"t_done_unix":1779712729.2712414},{"request_id":"1324964:3:1328499:384","session_id":"1324964","input_length":5234,"t_proxy_recv":281865.223324885,"t_decision_unix":1779712730.237266,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":5120,"affinity_cache_ratio":0.9782193351165457,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":5120,"estimated_new_tokens":114,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":888,"cache_hit":5120,"new_prefill":114,"score_linear":50010.0,"score_lmetric":114},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":5234,"score_linear":62618.0,"score_lmetric":5234},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1083,"cache_hit":0,"new_prefill":5234,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":94823,"ongoing_decode_tokens":0,"pending_prefill_tokens":68199,"num_requests":2,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":5234,"score_linear":94823.0,"score_lmetric":146866},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":5234,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":5234,"score_linear":33182.0,"score_lmetric":5234},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":5234,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":5234,"score_linear":81008.0,"score_lmetric":5234}],"chosen_score_linear":50010.0,"chosen_score_lmetric":114,"t_first_token":281865.25708241,"t_first_token_unix":1779712730.2710257,"t_done":281865.64136812,"t_done_unix":1779712730.6553116},{"request_id":"1317598:3:1328678:385","session_id":"1317598","input_length":81827,"t_proxy_recv":281865.900312152,"t_decision_unix":1779712730.9142535,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":81408,"affinity_cache_ratio":0.9948794407713836,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":81408,"estimated_new_tokens":419,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":888,"cache_hit":0,"new_prefill":81827,"score_linear":55130.0,"score_lmetric":81827},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":81827,"score_linear":62618.0,"score_lmetric":81827},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1083,"cache_hit":0,"new_prefill":81827,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":94823,"ongoing_decode_tokens":0,"pending_prefill_tokens":68199,"num_requests":2,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":81827,"score_linear":94823.0,"score_lmetric":300052},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":81408,"new_prefill":419,"score_linear":-81408.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":81827,"score_linear":33182.0,"score_lmetric":81827},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":81827,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":81827,"score_linear":81008.0,"score_lmetric":81827}],"chosen_score_linear":-81408.0,"chosen_score_lmetric":0,"t_first_token":281866.358589489,"t_first_token_unix":1779712731.372533,"t_done":281867.811834995,"t_done_unix":1779712732.8257785},{"request_id":"1279412:17:1326166:377","session_id":"1279412","input_length":81008,"t_proxy_recv":281856.890438061,"t_decision_unix":1779712721.9043794,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":75776,"affinity_cache_ratio":0.9354137862927119,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75776,"estimated_new_tokens":5232,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":887,"cache_hit":0,"new_prefill":81008,"score_linear":55130.0,"score_lmetric":81008},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":81008,"score_linear":62618.0,"score_lmetric":81008},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":81008,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":66703,"ongoing_decode_tokens":0,"pending_prefill_tokens":66703,"num_requests":1,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":81008,"score_linear":66703.0,"score_lmetric":147711},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81220,"ongoing_decode_tokens":81220,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1057,"cache_hit":0,"new_prefill":81008,"score_linear":81220.0,"score_lmetric":81008},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":81008,"score_linear":33182.0,"score_lmetric":81008},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":81008,"score_linear":191009.0,"score_lmetric":162016},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":945,"cache_hit":75776,"new_prefill":5232,"score_linear":-75776.0,"score_lmetric":0}],"chosen_score_linear":-75776.0,"chosen_score_lmetric":0,"t_first_token":281859.505107943,"t_first_token_unix":1779712724.5190513,"t_done":281868.311403978,"t_done_unix":1779712733.3253477},{"request_id":"1329110:1:1329110:389","session_id":"1329110","input_length":9292,"t_proxy_recv":281867.43585279,"t_decision_unix":1779712732.449794,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":9292,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":888,"cache_hit":0,"new_prefill":9292,"score_linear":55130.0,"score_lmetric":9292},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":9292,"score_linear":62618.0,"score_lmetric":9292},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1083,"cache_hit":0,"new_prefill":9292,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":94823,"ongoing_decode_tokens":0,"pending_prefill_tokens":68199,"num_requests":2,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":9292,"score_linear":94823.0,"score_lmetric":154982},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81827,"ongoing_decode_tokens":81827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":9292,"score_linear":81827.0,"score_lmetric":9292},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":67446,"ongoing_decode_tokens":33182,"pending_prefill_tokens":7640,"num_requests":2,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":9292,"score_linear":67446.0,"score_lmetric":33864},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":36880,"ongoing_decode_tokens":0,"pending_prefill_tokens":36880,"num_requests":1,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":9292,"score_linear":36880.0,"score_lmetric":46172},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":9292,"score_linear":81008.0,"score_lmetric":9292}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281868.142660052,"t_first_token_unix":1779712733.1566033,"t_done":281868.455080099,"t_done_unix":1779712733.4690237},{"request_id":"1324964:4:1329201:390","session_id":"1324964","input_length":5414,"t_proxy_recv":281867.746258343,"t_decision_unix":1779712732.7601993,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":5120,"affinity_cache_ratio":0.9456963428149243,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":5120,"estimated_new_tokens":294,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":888,"cache_hit":5120,"new_prefill":294,"score_linear":50010.0,"score_lmetric":294},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":5414,"score_linear":62618.0,"score_lmetric":5414},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":9292,"ongoing_decode_tokens":0,"pending_prefill_tokens":9292,"num_requests":1,"active_p_offloads":0,"cached_blocks":1083,"cache_hit":0,"new_prefill":5414,"score_linear":9292.0,"score_lmetric":14706},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":94823,"ongoing_decode_tokens":0,"pending_prefill_tokens":68199,"num_requests":2,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":5414,"score_linear":94823.0,"score_lmetric":147226},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81827,"ongoing_decode_tokens":81827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":5414,"score_linear":81827.0,"score_lmetric":5414},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":67446,"ongoing_decode_tokens":33182,"pending_prefill_tokens":7640,"num_requests":2,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":5414,"score_linear":67446.0,"score_lmetric":26108},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":36880,"ongoing_decode_tokens":0,"pending_prefill_tokens":36880,"num_requests":1,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":5414,"score_linear":36880.0,"score_lmetric":42294},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":5414,"score_linear":81008.0,"score_lmetric":5414}],"chosen_score_linear":50010.0,"chosen_score_lmetric":294,"t_first_token":281867.799956074,"t_first_token_unix":1779712732.8138993,"t_done":281868.509276224,"t_done_unix":1779712733.5232198},{"request_id":"1325987:2:1329336:391","session_id":"1325987","input_length":7423,"t_proxy_recv":281868.283997944,"t_decision_unix":1779712733.2979393,"policy":"unified","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":4,"affinity_cache_hit":1536,"affinity_cache_ratio":0.20692442408729625,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":7423,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":60544,"ongoing_decode_tokens":60544,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":888,"cache_hit":0,"new_prefill":7423,"score_linear":60544.0,"score_lmetric":14846},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":7423,"score_linear":62618.0,"score_lmetric":7423},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":9292,"ongoing_decode_tokens":9292,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1083,"cache_hit":1536,"new_prefill":5887,"score_linear":7756.0,"score_lmetric":5887},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":94823,"ongoing_decode_tokens":0,"pending_prefill_tokens":68199,"num_requests":2,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":7423,"score_linear":94823.0,"score_lmetric":151244},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1060,"cache_hit":0,"new_prefill":7423,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":67446,"ongoing_decode_tokens":33182,"pending_prefill_tokens":7640,"num_requests":2,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":7423,"score_linear":67446.0,"score_lmetric":30126},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":36880,"ongoing_decode_tokens":0,"pending_prefill_tokens":36880,"num_requests":1,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":7423,"score_linear":36880.0,"score_lmetric":44303},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":7423,"score_linear":81008.0,"score_lmetric":7423}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281868.792652443,"t_first_token_unix":1779712733.8065963,"t_done":281868.996297215,"t_done_unix":1779712734.0102408},{"request_id":"1322225:1:1322225:358","session_id":"1322225","input_length":33182,"t_proxy_recv":281843.00039721,"t_decision_unix":1779712708.0143385,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":33182,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":50450,"num_requests":1,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":33182,"score_linear":54546.0,"score_lmetric":83632},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":33182,"score_linear":62618.0,"score_lmetric":33182},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":33182,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21471,"ongoing_decode_tokens":21471,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1012,"cache_hit":0,"new_prefill":33182,"score_linear":21471.0,"score_lmetric":33182},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":33182,"score_linear":79663.0,"score_lmetric":112333},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":858,"cache_hit":0,"new_prefill":33182,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188808,"ongoing_decode_tokens":188808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":33182,"score_linear":188808.0,"score_lmetric":66364},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":72402,"ongoing_decode_tokens":72402,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":914,"cache_hit":0,"new_prefill":33182,"score_linear":72402.0,"score_lmetric":33182}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281847.689556098,"t_first_token_unix":1779712712.7034993,"t_done":281870.328360349,"t_done_unix":1779712735.3423064},{"request_id":"1325541:1:1325541:374","session_id":"1325541","input_length":66703,"t_proxy_recv":281854.660769649,"t_decision_unix":1779712719.6747108,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":66703,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":59442,"ongoing_decode_tokens":59442,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":878,"cache_hit":0,"new_prefill":66703,"score_linear":59442.0,"score_lmetric":133406},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":66703,"score_linear":62618.0,"score_lmetric":66703},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":66703,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":66703,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":66703,"score_linear":79663.0,"score_lmetric":66703},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":66703,"score_linear":33182.0,"score_lmetric":66703},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":66703,"score_linear":121968.0,"score_lmetric":66703},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":945,"cache_hit":0,"new_prefill":66703,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281870.433271668,"t_first_token_unix":1779712735.4472153,"t_done":281870.766457619,"t_done_unix":1779712735.780401},{"request_id":"1329470:1:1329470:392","session_id":"1329470","input_length":10847,"t_proxy_recv":281868.775529005,"t_decision_unix":1779712733.78947,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":10847,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":888,"cache_hit":0,"new_prefill":10847,"score_linear":55130.0,"score_lmetric":10847},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":10847,"score_linear":62618.0,"score_lmetric":10847},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1101,"cache_hit":0,"new_prefill":10847,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":94823,"ongoing_decode_tokens":0,"pending_prefill_tokens":68199,"num_requests":2,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":10847,"score_linear":94823.0,"score_lmetric":158092},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":7423,"ongoing_decode_tokens":0,"pending_prefill_tokens":7423,"num_requests":1,"active_p_offloads":0,"cached_blocks":1060,"cache_hit":0,"new_prefill":10847,"score_linear":7423.0,"score_lmetric":18270},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":67446,"ongoing_decode_tokens":33182,"pending_prefill_tokens":7640,"num_requests":2,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":10847,"score_linear":67446.0,"score_lmetric":36974},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":36880,"ongoing_decode_tokens":0,"pending_prefill_tokens":36880,"num_requests":1,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":10847,"score_linear":36880.0,"score_lmetric":47727},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":958,"cache_hit":0,"new_prefill":10847,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281869.636317933,"t_first_token_unix":1779712734.650262,"t_done":281871.153763824,"t_done_unix":1779712736.167707},{"request_id":"1324964:5:1330169:397","session_id":"1324964","input_length":6691,"t_proxy_recv":281871.140380151,"t_decision_unix":1779712736.1543217,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":5120,"affinity_cache_ratio":0.7652069944701838,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":5120,"estimated_new_tokens":1571,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":888,"cache_hit":5120,"new_prefill":1571,"score_linear":50010.0,"score_lmetric":1571},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":6691,"score_linear":62618.0,"score_lmetric":6691},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10847,"ongoing_decode_tokens":10847,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1101,"cache_hit":0,"new_prefill":6691,"score_linear":10847.0,"score_lmetric":6691},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":28120,"ongoing_decode_tokens":28120,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1203,"cache_hit":0,"new_prefill":6691,"score_linear":28120.0,"score_lmetric":6691},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":6691,"score_linear":90605.0,"score_lmetric":7184},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34264,"ongoing_decode_tokens":34264,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":934,"cache_hit":0,"new_prefill":6691,"score_linear":34264.0,"score_lmetric":6691},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":36880,"ongoing_decode_tokens":0,"pending_prefill_tokens":36880,"num_requests":1,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":6691,"score_linear":36880.0,"score_lmetric":43571},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":11725,"ongoing_decode_tokens":0,"pending_prefill_tokens":11725,"num_requests":1,"active_p_offloads":0,"cached_blocks":958,"cache_hit":0,"new_prefill":6691,"score_linear":11725.0,"score_lmetric":18416}],"chosen_score_linear":50010.0,"chosen_score_lmetric":1571,"t_first_token":281871.335281689,"t_first_token_unix":1779712736.349225,"t_done":281871.698833022,"t_done_unix":1779712736.7127762},{"request_id":"1328753:1:1328753:387","session_id":"1328753","input_length":36880,"t_proxy_recv":281866.173014612,"t_decision_unix":1779712731.1869557,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":36880,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":888,"cache_hit":0,"new_prefill":36880,"score_linear":55130.0,"score_lmetric":36880},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":36880,"score_linear":62618.0,"score_lmetric":36880},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1083,"cache_hit":0,"new_prefill":36880,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":94823,"ongoing_decode_tokens":0,"pending_prefill_tokens":68199,"num_requests":2,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":36880,"score_linear":94823.0,"score_lmetric":210158},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81827,"ongoing_decode_tokens":0,"pending_prefill_tokens":419,"num_requests":1,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":36880,"score_linear":81827.0,"score_lmetric":37299},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":36880,"score_linear":33182.0,"score_lmetric":36880},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":36880,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":36880,"score_linear":81008.0,"score_lmetric":36880}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281871.728136674,"t_first_token_unix":1779712736.74208,"t_done":281872.113786432,"t_done_unix":1779712737.1277294},{"request_id":"1268861:5:1329060:388","session_id":"1268861","input_length":34264,"t_proxy_recv":281867.30801325,"t_decision_unix":1779712732.3219543,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":26624,"affinity_cache_ratio":0.7770254494513191,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":26624,"estimated_new_tokens":7640,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":888,"cache_hit":0,"new_prefill":34264,"score_linear":55130.0,"score_lmetric":34264},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":34264,"score_linear":62618.0,"score_lmetric":34264},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1083,"cache_hit":0,"new_prefill":34264,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":94823,"ongoing_decode_tokens":0,"pending_prefill_tokens":68199,"num_requests":2,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":34264,"score_linear":94823.0,"score_lmetric":204926},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81827,"ongoing_decode_tokens":81827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":21504,"new_prefill":12760,"score_linear":60323.0,"score_lmetric":12760},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":26624,"new_prefill":7640,"score_linear":6558.0,"score_lmetric":7640},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":36880,"ongoing_decode_tokens":0,"pending_prefill_tokens":36880,"num_requests":1,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":34264,"score_linear":36880.0,"score_lmetric":71144},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":34264,"score_linear":81008.0,"score_lmetric":34264}],"chosen_score_linear":6558.0,"chosen_score_lmetric":7640,"t_first_token":281868.933916797,"t_first_token_unix":1779712733.9478602,"t_done":281872.134916142,"t_done_unix":1779712737.148859},{"request_id":"1330130:1:1330130:396","session_id":"1330130","input_length":11725,"t_proxy_recv":281871.039675322,"t_decision_unix":1779712736.0536165,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":11725,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":888,"cache_hit":0,"new_prefill":11725,"score_linear":55130.0,"score_lmetric":11725},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":11725,"score_linear":62618.0,"score_lmetric":11725},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10847,"ongoing_decode_tokens":10847,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1101,"cache_hit":0,"new_prefill":11725,"score_linear":10847.0,"score_lmetric":11725},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":28120,"ongoing_decode_tokens":28120,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1203,"cache_hit":0,"new_prefill":11725,"score_linear":28120.0,"score_lmetric":11725},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":11725,"score_linear":90605.0,"score_lmetric":12218},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34264,"ongoing_decode_tokens":34264,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":934,"cache_hit":0,"new_prefill":11725,"score_linear":34264.0,"score_lmetric":11725},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":36880,"ongoing_decode_tokens":0,"pending_prefill_tokens":36880,"num_requests":1,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":11725,"score_linear":36880.0,"score_lmetric":48605},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":958,"cache_hit":0,"new_prefill":11725,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281871.983974059,"t_first_token_unix":1779712736.9979174,"t_done":281872.441598212,"t_done_unix":1779712737.4555416},{"request_id":"1316848:8:1326380:379","session_id":"1316848","input_length":28120,"t_proxy_recv":281857.655868789,"t_decision_unix":1779712722.6698098,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":26624,"affinity_cache_ratio":0.9467994310099573,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":26624,"estimated_new_tokens":1496,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":887,"cache_hit":0,"new_prefill":28120,"score_linear":55130.0,"score_lmetric":28120},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":28120,"score_linear":62618.0,"score_lmetric":28120},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":9513,"ongoing_decode_tokens":0,"pending_prefill_tokens":9513,"num_requests":1,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":28120,"score_linear":9513.0,"score_lmetric":37633},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":66703,"ongoing_decode_tokens":0,"pending_prefill_tokens":66703,"num_requests":1,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":26624,"new_prefill":1496,"score_linear":40079.0,"score_lmetric":68199},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81220,"ongoing_decode_tokens":81220,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1057,"cache_hit":0,"new_prefill":28120,"score_linear":81220.0,"score_lmetric":28120},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":28120,"score_linear":33182.0,"score_lmetric":28120},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":780,"cache_hit":0,"new_prefill":28120,"score_linear":191009.0,"score_lmetric":56240},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":81008,"ongoing_decode_tokens":0,"pending_prefill_tokens":5232,"num_requests":1,"active_p_offloads":0,"cached_blocks":945,"cache_hit":0,"new_prefill":28120,"score_linear":81008.0,"score_lmetric":33352}],"chosen_score_linear":40079.0,"chosen_score_lmetric":68199,"t_first_token":281870.434655569,"t_first_token_unix":1779712735.4485989,"t_done":281872.473897114,"t_done_unix":1779712737.4878402},{"request_id":"1328753:2:1330033:395","session_id":"1328753","input_length":37007,"t_proxy_recv":281872.121831399,"t_decision_unix":1779712737.135773,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":36864,"affinity_cache_ratio":0.9961358661874781,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":36864,"estimated_new_tokens":143,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":37007,"score_linear":55130.0,"score_lmetric":37007},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":37007,"score_linear":62618.0,"score_lmetric":37007},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":37007,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":28120,"ongoing_decode_tokens":28120,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1203,"cache_hit":0,"new_prefill":37007,"score_linear":28120.0,"score_lmetric":37007},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":37007,"score_linear":90605.0,"score_lmetric":37500},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34264,"ongoing_decode_tokens":34264,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":934,"cache_hit":0,"new_prefill":37007,"score_linear":34264.0,"score_lmetric":37007},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":905,"cache_hit":36864,"new_prefill":143,"score_linear":-36864.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":11725,"ongoing_decode_tokens":11725,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":958,"cache_hit":0,"new_prefill":37007,"score_linear":11725.0,"score_lmetric":37007}],"chosen_score_linear":-36864.0,"chosen_score_lmetric":0,"t_first_token":281872.21715706,"t_first_token_unix":1779712737.2311,"t_done":281872.673560664,"t_done_unix":1779712737.6875036},{"request_id":"1316848:9:1328693:386","session_id":"1316848","input_length":28615,"t_proxy_recv":281872.480311079,"t_decision_unix":1779712737.4942524,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":28160,"affinity_cache_ratio":0.9840992486458151,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":28160,"estimated_new_tokens":455,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":28615,"score_linear":55130.0,"score_lmetric":28615},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":28615,"score_linear":62618.0,"score_lmetric":28615},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":28615,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1206,"cache_hit":28160,"new_prefill":455,"score_linear":-28160.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":28615,"score_linear":90605.0,"score_lmetric":29108},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":28615,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37007,"ongoing_decode_tokens":37007,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":28615,"score_linear":37007.0,"score_lmetric":28615},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":981,"cache_hit":0,"new_prefill":28615,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-28160.0,"chosen_score_lmetric":0,"t_first_token":281872.593986095,"t_first_token_unix":1779712737.60793,"t_done":281874.284788401,"t_done_unix":1779712739.2987313},{"request_id":"1330881:1:1330881:400","session_id":"1330881","input_length":3866,"t_proxy_recv":281873.798211336,"t_decision_unix":1779712738.8121524,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3866,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":3866,"score_linear":55130.0,"score_lmetric":3866},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":3866,"score_linear":62618.0,"score_lmetric":3866},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":25688,"ongoing_decode_tokens":0,"pending_prefill_tokens":10840,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":3866,"score_linear":25688.0,"score_lmetric":14706},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":28615,"ongoing_decode_tokens":28615,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1206,"cache_hit":0,"new_prefill":3866,"score_linear":28615.0,"score_lmetric":3866},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":3866,"score_linear":90605.0,"score_lmetric":4359},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":3866,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":3866,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":981,"cache_hit":0,"new_prefill":3866,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281874.039164263,"t_first_token_unix":1779712739.0531077,"t_done":281874.403522985,"t_done_unix":1779712739.4174662},{"request_id":"1326192:3:1330643:399","session_id":"1326192","input_length":25688,"t_proxy_recv":281872.940238145,"t_decision_unix":1779712737.9541793,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":14848,"affinity_cache_ratio":0.5780130800373715,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":14848,"estimated_new_tokens":10840,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":25688,"score_linear":55130.0,"score_lmetric":25688},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":25688,"score_linear":62618.0,"score_lmetric":25688},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":14848,"new_prefill":10840,"score_linear":-14848.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":28615,"ongoing_decode_tokens":28615,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1206,"cache_hit":0,"new_prefill":25688,"score_linear":28615.0,"score_lmetric":25688},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":25688,"score_linear":90605.0,"score_lmetric":26181},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":25688,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":25688,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":981,"cache_hit":0,"new_prefill":25688,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-14848.0,"chosen_score_lmetric":0,"t_first_token":281874.69758332,"t_first_token_unix":1779712739.711527,"t_done":281874.984394481,"t_done_unix":1779712739.9983377},{"request_id":"1326192:4:1331370:401","session_id":"1326192","input_length":27049,"t_proxy_recv":281875.572195765,"t_decision_unix":1779712740.586137,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":25600,"affinity_cache_ratio":0.9464305519612555,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":25600,"estimated_new_tokens":1449,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":27049,"score_linear":55130.0,"score_lmetric":27049},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":27049,"score_linear":62618.0,"score_lmetric":27049},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1143,"cache_hit":25600,"new_prefill":1449,"score_linear":-25600.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":28977,"ongoing_decode_tokens":28977,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1207,"cache_hit":0,"new_prefill":27049,"score_linear":28977.0,"score_lmetric":27049},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":27049,"score_linear":90605.0,"score_lmetric":27542},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":27049,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":27049,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":988,"cache_hit":0,"new_prefill":27049,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-25600.0,"chosen_score_lmetric":0,"t_first_token":281875.917673256,"t_first_token_unix":1779712740.9316165,"t_done":281876.734698561,"t_done_unix":1779712741.7486417},{"request_id":"1329110:2:1331399:402","session_id":"1329110","input_length":12781,"t_proxy_recv":281875.658624237,"t_decision_unix":1779712740.6725655,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":9216,"affinity_cache_ratio":0.7210703387841327,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9216,"estimated_new_tokens":3565,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":12781,"score_linear":55130.0,"score_lmetric":12781},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":12781,"score_linear":62618.0,"score_lmetric":12781},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":27049,"ongoing_decode_tokens":0,"pending_prefill_tokens":1449,"num_requests":1,"active_p_offloads":0,"cached_blocks":1143,"cache_hit":9216,"new_prefill":3565,"score_linear":17833.0,"score_lmetric":5014},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":28977,"ongoing_decode_tokens":28977,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1207,"cache_hit":0,"new_prefill":12781,"score_linear":28977.0,"score_lmetric":12781},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":12781,"score_linear":90605.0,"score_lmetric":13274},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":12781,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":12781,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":988,"cache_hit":0,"new_prefill":12781,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":17833.0,"chosen_score_lmetric":5014,"t_first_token":281876.302277489,"t_first_token_unix":1779712741.3162215,"t_done":281876.808146014,"t_done_unix":1779712741.8220892},{"request_id":"1330130:2:1331784:403","session_id":"1330130","input_length":14067,"t_proxy_recv":281876.845903682,"t_decision_unix":1779712741.8598452,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":11776,"affinity_cache_ratio":0.8371365607450061,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":2291,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":14067,"score_linear":55130.0,"score_lmetric":14067},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":14067,"score_linear":62618.0,"score_lmetric":14067},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":0,"new_prefill":14067,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":28977,"ongoing_decode_tokens":28977,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1207,"cache_hit":0,"new_prefill":14067,"score_linear":28977.0,"score_lmetric":14067},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":14067,"score_linear":90605.0,"score_lmetric":14560},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":14067,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":14067,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":988,"cache_hit":11776,"new_prefill":2291,"score_linear":-11776.0,"score_lmetric":0}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":0,"t_first_token":281877.159109499,"t_first_token_unix":1779712742.1730528,"t_done":281877.299212045,"t_done_unix":1779712742.3131552},{"request_id":"1332021:1:1332021:407","session_id":"1332021","input_length":12421,"t_proxy_recv":281877.643269655,"t_decision_unix":1779712742.6572113,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":12421,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":12421,"score_linear":55130.0,"score_lmetric":12421},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":12421,"score_linear":62618.0,"score_lmetric":12421},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":21680,"ongoing_decode_tokens":0,"pending_prefill_tokens":21680,"num_requests":1,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":0,"new_prefill":12421,"score_linear":21680.0,"score_lmetric":34101},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":102211,"ongoing_decode_tokens":28977,"pending_prefill_tokens":6674,"num_requests":2,"active_p_offloads":0,"cached_blocks":1207,"cache_hit":0,"new_prefill":12421,"score_linear":102211.0,"score_lmetric":38190},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":12421,"score_linear":90605.0,"score_lmetric":12914},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":12421,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":71777,"ongoing_decode_tokens":0,"pending_prefill_tokens":2657,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":12421,"score_linear":71777.0,"score_lmetric":15078},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":12421,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281878.66101652,"t_first_token_unix":1779712743.6749604,"t_done":281878.918718296,"t_done_unix":1779712743.9326615},{"request_id":"1325541:2:1331846:404","session_id":"1325541","input_length":73234,"t_proxy_recv":281877.04703233,"t_decision_unix":1779712742.0609736,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":66560,"affinity_cache_ratio":0.9088674659311249,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":66560,"estimated_new_tokens":6674,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":73234,"score_linear":55130.0,"score_lmetric":73234},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":73234,"score_linear":62618.0,"score_lmetric":73234},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":0,"new_prefill":73234,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":28977,"ongoing_decode_tokens":28977,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1207,"cache_hit":66560,"new_prefill":6674,"score_linear":-37583.0,"score_lmetric":6674},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":73234,"score_linear":90605.0,"score_lmetric":73727},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":73234,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":73234,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":14067,"ongoing_decode_tokens":0,"pending_prefill_tokens":2291,"num_requests":1,"active_p_offloads":0,"cached_blocks":988,"cache_hit":0,"new_prefill":73234,"score_linear":14067.0,"score_lmetric":75525}],"chosen_score_linear":-37583.0,"chosen_score_lmetric":6674,"t_first_token":281879.957289296,"t_first_token_unix":1779712744.9712334,"t_done":281880.048742146,"t_done_unix":1779712745.0626857},{"request_id":"1332472:1:1332472:410","session_id":"1332472","input_length":1810,"t_proxy_recv":281879.300103539,"t_decision_unix":1779712744.314045,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1810,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":1810,"score_linear":55130.0,"score_lmetric":1810},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":1810,"score_linear":62618.0,"score_lmetric":1810},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":33446,"ongoing_decode_tokens":0,"pending_prefill_tokens":22694,"num_requests":2,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":0,"new_prefill":1810,"score_linear":33446.0,"score_lmetric":49008},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":102211,"ongoing_decode_tokens":28977,"pending_prefill_tokens":6674,"num_requests":2,"active_p_offloads":0,"cached_blocks":1207,"cache_hit":0,"new_prefill":1810,"score_linear":102211.0,"score_lmetric":16968},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":1810,"score_linear":172747.0,"score_lmetric":5050},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":1810,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":71777,"ongoing_decode_tokens":71777,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":1810,"score_linear":71777.0,"score_lmetric":1810},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":1810,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281879.421861739,"t_first_token_unix":1779712744.4358056,"t_done":281880.1233816,"t_done_unix":1779712745.1373248},{"request_id":"1329470:2:1332133:409","session_id":"1329470","input_length":11766,"t_proxy_recv":281878.01188657,"t_decision_unix":1779712743.0258284,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9138194798572157,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":10752,"estimated_new_tokens":1014,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":11766,"score_linear":55130.0,"score_lmetric":11766},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":11766,"score_linear":62618.0,"score_lmetric":11766},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":21680,"ongoing_decode_tokens":0,"pending_prefill_tokens":21680,"num_requests":1,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":10752,"new_prefill":1014,"score_linear":10928.0,"score_lmetric":22694},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":102211,"ongoing_decode_tokens":28977,"pending_prefill_tokens":6674,"num_requests":2,"active_p_offloads":0,"cached_blocks":1207,"cache_hit":0,"new_prefill":11766,"score_linear":102211.0,"score_lmetric":36880},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":11766,"score_linear":172747.0,"score_lmetric":24962},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":11766,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":71777,"ongoing_decode_tokens":0,"pending_prefill_tokens":2657,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":11766,"score_linear":71777.0,"score_lmetric":14423},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":12421,"ongoing_decode_tokens":0,"pending_prefill_tokens":12421,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":11766,"score_linear":12421.0,"score_lmetric":24187}],"chosen_score_linear":10928.0,"chosen_score_lmetric":22694,"t_first_token":281880.039667991,"t_first_token_unix":1779712745.0536113,"t_done":281880.256206682,"t_done_unix":1779712745.2701497},{"request_id":"1316848:10:1330509:398","session_id":"1316848","input_length":28977,"t_proxy_recv":281874.29265642,"t_decision_unix":1779712739.306598,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":28672,"affinity_cache_ratio":0.9894744107395521,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":28672,"estimated_new_tokens":305,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":28977,"score_linear":55130.0,"score_lmetric":28977},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":28977,"score_linear":62618.0,"score_lmetric":28977},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":25688,"ongoing_decode_tokens":0,"pending_prefill_tokens":10840,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":28977,"score_linear":25688.0,"score_lmetric":39817},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1207,"cache_hit":28672,"new_prefill":305,"score_linear":-28672.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":28977,"score_linear":90605.0,"score_lmetric":29470},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":28977,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":28977,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":3866,"ongoing_decode_tokens":3866,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":981,"cache_hit":0,"new_prefill":28977,"score_linear":3866.0,"score_lmetric":28977}],"chosen_score_linear":-28672.0,"chosen_score_lmetric":0,"t_first_token":281874.387363249,"t_first_token_unix":1779712739.4013066,"t_done":281880.338317517,"t_done_unix":1779712745.3522627},{"request_id":"1329110:3:1332476:411","session_id":"1329110","input_length":13479,"t_proxy_recv":281879.325601154,"t_decision_unix":1779712744.3395426,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":12800,"affinity_cache_ratio":0.9496253431263447,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":679,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":13479,"score_linear":55130.0,"score_lmetric":13479},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":13479,"score_linear":62618.0,"score_lmetric":13479},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":33446,"ongoing_decode_tokens":0,"pending_prefill_tokens":22694,"num_requests":2,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":12800,"new_prefill":679,"score_linear":20646.0,"score_lmetric":46746},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":102211,"ongoing_decode_tokens":28977,"pending_prefill_tokens":6674,"num_requests":2,"active_p_offloads":0,"cached_blocks":1207,"cache_hit":0,"new_prefill":13479,"score_linear":102211.0,"score_lmetric":40306},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":13479,"score_linear":172747.0,"score_lmetric":28388},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":1810,"ongoing_decode_tokens":0,"pending_prefill_tokens":1810,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":13479,"score_linear":1810.0,"score_lmetric":15289},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":71777,"ongoing_decode_tokens":71777,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":13479,"score_linear":71777.0,"score_lmetric":13479},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":13479,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":20646.0,"chosen_score_lmetric":46746,"t_first_token":281880.146933916,"t_first_token_unix":1779712745.1608772,"t_done":281880.678559756,"t_done_unix":1779712745.692503},{"request_id":"1269373:16:1331896:405","session_id":"1269373","input_length":71777,"t_proxy_recv":281877.241150208,"t_decision_unix":1779712742.2550917,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":69120,"affinity_cache_ratio":0.9629825710185714,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":69120,"estimated_new_tokens":2657,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":7680,"new_prefill":64097,"score_linear":47450.0,"score_lmetric":64097},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":71777,"score_linear":62618.0,"score_lmetric":71777},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":0,"new_prefill":71777,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":102211,"ongoing_decode_tokens":28977,"pending_prefill_tokens":6674,"num_requests":2,"active_p_offloads":0,"cached_blocks":1207,"cache_hit":7680,"new_prefill":64097,"score_linear":94531.0,"score_lmetric":141542},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":71777,"score_linear":90605.0,"score_lmetric":72270},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":71777,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":905,"cache_hit":69120,"new_prefill":2657,"score_linear":-69120.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":14067,"ongoing_decode_tokens":14067,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":988,"cache_hit":7680,"new_prefill":64097,"score_linear":6387.0,"score_lmetric":64097}],"chosen_score_linear":-69120.0,"chosen_score_lmetric":0,"t_first_token":281878.406315998,"t_first_token_unix":1779712743.4202595,"t_done":281881.053432439,"t_done_unix":1779712746.067376},{"request_id":"1332013:1:1332013:406","session_id":"1332013","input_length":21680,"t_proxy_recv":281877.627813027,"t_decision_unix":1779712742.6417546,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":21680,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":21680,"score_linear":55130.0,"score_lmetric":21680},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":12800,"new_prefill":8880,"score_linear":49818.0,"score_lmetric":8880},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":0,"new_prefill":21680,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":102211,"ongoing_decode_tokens":28977,"pending_prefill_tokens":6674,"num_requests":2,"active_p_offloads":0,"cached_blocks":1207,"cache_hit":12800,"new_prefill":8880,"score_linear":89411.0,"score_lmetric":31108},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":21680,"score_linear":90605.0,"score_lmetric":22173},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":21680,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":71777,"ongoing_decode_tokens":0,"pending_prefill_tokens":2657,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":21680,"score_linear":71777.0,"score_lmetric":24337},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":21680,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281880.037569898,"t_first_token_unix":1779712745.051514,"t_done":281881.419306046,"t_done_unix":1779712746.4332492},{"request_id":"1332754:1:1332754:412","session_id":"1332754","input_length":19766,"t_proxy_recv":281880.327989359,"t_decision_unix":1779712745.3419297,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":19766,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":19766,"score_linear":55130.0,"score_lmetric":19766},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":19766,"score_linear":62618.0,"score_lmetric":19766},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":35159,"ongoing_decode_tokens":35159,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1154,"cache_hit":0,"new_prefill":19766,"score_linear":35159.0,"score_lmetric":39532},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":28977,"ongoing_decode_tokens":28977,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1220,"cache_hit":0,"new_prefill":19766,"score_linear":28977.0,"score_lmetric":19766},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":19766,"score_linear":172747.0,"score_lmetric":40962},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":19766,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":71777,"ongoing_decode_tokens":71777,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":19766,"score_linear":71777.0,"score_lmetric":19766},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":19766,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281882.386924254,"t_first_token_unix":1779712747.4008677,"t_done":281883.578984943,"t_done_unix":1779712748.5929284},{"request_id":"1316848:11:1333994:416","session_id":"1316848","input_length":29656,"t_proxy_recv":281884.703294757,"t_decision_unix":1779712749.717236,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":29184,"affinity_cache_ratio":0.9840841650930672,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":29184,"estimated_new_tokens":472,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":29656,"score_linear":55130.0,"score_lmetric":29656},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":29656,"score_linear":62618.0,"score_lmetric":29656},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":29656,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1221,"cache_hit":29184,"new_prefill":472,"score_linear":-29184.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":29656,"score_linear":172747.0,"score_lmetric":60742},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":29656,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":910,"cache_hit":0,"new_prefill":29656,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":82488,"ongoing_decode_tokens":82488,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":29656,"score_linear":82488.0,"score_lmetric":29656}],"chosen_score_linear":-29184.0,"chosen_score_lmetric":0,"t_first_token":281884.821805219,"t_first_token_unix":1779712749.8357487,"t_done":281885.109912713,"t_done_unix":1779712750.1238556},{"request_id":"1279412:18:1332945:413","session_id":"1279412","input_length":82488,"t_proxy_recv":281881.164622754,"t_decision_unix":1779712746.1785643,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":81408,"affinity_cache_ratio":0.9869071864998545,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":81408,"estimated_new_tokens":1080,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":82488,"score_linear":55130.0,"score_lmetric":82488},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":82488,"score_linear":62618.0,"score_lmetric":82488},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":21680,"ongoing_decode_tokens":21680,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1155,"cache_hit":0,"new_prefill":82488,"score_linear":21680.0,"score_lmetric":82488},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1221,"cache_hit":0,"new_prefill":82488,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":82488,"score_linear":172747.0,"score_lmetric":166406},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":82488,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":910,"cache_hit":0,"new_prefill":82488,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":19766,"ongoing_decode_tokens":0,"pending_prefill_tokens":19766,"num_requests":1,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":81408,"new_prefill":1080,"score_linear":-61642.0,"score_lmetric":20846}],"chosen_score_linear":-61642.0,"chosen_score_lmetric":20846,"t_first_token":281882.712458681,"t_first_token_unix":1779712747.7264023,"t_done":281885.622968887,"t_done_unix":1779712750.636912},{"request_id":"1334223:1:1334223:417","session_id":"1334223","input_length":23689,"t_proxy_recv":281885.574175756,"t_decision_unix":1779712750.5881174,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":23689,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":23689,"score_linear":55130.0,"score_lmetric":23689},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":23689,"score_linear":62618.0,"score_lmetric":23689},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":23689,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":0,"new_prefill":23689,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":23689,"score_linear":172747.0,"score_lmetric":48808},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":23689,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":910,"cache_hit":0,"new_prefill":23689,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":82488,"ongoing_decode_tokens":82488,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":23689,"score_linear":82488.0,"score_lmetric":23689}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281888.233732993,"t_first_token_unix":1779712753.2476766,"t_done":281888.832278924,"t_done_unix":1779712753.8462238},{"request_id":"1316848:12:1334743:418","session_id":"1316848","input_length":44664,"t_proxy_recv":281887.542674476,"t_decision_unix":1779712752.5566158,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":29696,"affinity_cache_ratio":0.6648755149561167,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":29696,"estimated_new_tokens":14968,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":44664,"score_linear":55130.0,"score_lmetric":44664},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":44664,"score_linear":62618.0,"score_lmetric":44664},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":44664,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":29696,"new_prefill":14968,"score_linear":-29696.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":44664,"score_linear":172747.0,"score_lmetric":90758},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":23689,"ongoing_decode_tokens":0,"pending_prefill_tokens":23689,"num_requests":1,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":44664,"score_linear":23689.0,"score_lmetric":68353},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":910,"cache_hit":0,"new_prefill":44664,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1056,"cache_hit":0,"new_prefill":44664,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-29696.0,"chosen_score_lmetric":0,"t_first_token":281891.386937566,"t_first_token_unix":1779712756.4008808,"t_done":281892.323036966,"t_done_unix":1779712757.3369803},{"request_id":"1314357:2:1335184:420","session_id":"1314357","input_length":61229,"t_proxy_recv":281888.907939434,"t_decision_unix":1779712753.9218807,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":56832,"affinity_cache_ratio":0.9281876235117347,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":56832,"estimated_new_tokens":4397,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":56832,"new_prefill":4397,"score_linear":-1702.0,"score_lmetric":4397},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":61229,"score_linear":62618.0,"score_lmetric":61229},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":61229,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":44664,"ongoing_decode_tokens":0,"pending_prefill_tokens":14968,"num_requests":1,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":0,"new_prefill":61229,"score_linear":44664.0,"score_lmetric":76197},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":218348,"ongoing_decode_tokens":0,"pending_prefill_tokens":1772,"num_requests":3,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":61229,"score_linear":218348.0,"score_lmetric":189003},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":998,"cache_hit":0,"new_prefill":61229,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":910,"cache_hit":0,"new_prefill":61229,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1056,"cache_hit":0,"new_prefill":61229,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-1702.0,"chosen_score_lmetric":4397,"t_first_token":281890.506128828,"t_first_token_unix":1779712755.5200725,"t_done":281893.466328395,"t_done_unix":1779712758.4802718},{"request_id":"1330881:2:1336222:422","session_id":"1330881","input_length":11989,"t_proxy_recv":281892.662191095,"t_decision_unix":1779712757.6761324,"policy":"unified","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":6,"affinity_cache_hit":3584,"affinity_cache_ratio":0.29894069563766784,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":11989,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":116359,"ongoing_decode_tokens":116359,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":11989,"score_linear":116359.0,"score_lmetric":23978},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":11989,"score_linear":62618.0,"score_lmetric":11989},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":11989,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1251,"cache_hit":0,"new_prefill":11989,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":218348,"ongoing_decode_tokens":0,"pending_prefill_tokens":1772,"num_requests":3,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":11989,"score_linear":218348.0,"score_lmetric":41283},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":998,"cache_hit":0,"new_prefill":11989,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":910,"cache_hit":0,"new_prefill":11989,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":87264,"ongoing_decode_tokens":87264,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1056,"cache_hit":3584,"new_prefill":8405,"score_linear":83680.0,"score_lmetric":8405}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281893.644912382,"t_first_token_unix":1779712758.6588562,"t_done":281893.8045355,"t_done_unix":1779712758.8184786},{"request_id":"1279412:19:1335444:421","session_id":"1279412","input_length":87264,"t_proxy_recv":281889.844829127,"t_decision_unix":1779712754.8587704,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":82432,"affinity_cache_ratio":0.9446277961129447,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":82432,"estimated_new_tokens":4832,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":116359,"ongoing_decode_tokens":55130,"pending_prefill_tokens":4397,"num_requests":2,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":87264,"score_linear":116359.0,"score_lmetric":183322},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":87264,"score_linear":62618.0,"score_lmetric":87264},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":87264,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":44664,"ongoing_decode_tokens":0,"pending_prefill_tokens":14968,"num_requests":1,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":0,"new_prefill":87264,"score_linear":44664.0,"score_lmetric":102232},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":218348,"ongoing_decode_tokens":0,"pending_prefill_tokens":1772,"num_requests":3,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":87264,"score_linear":218348.0,"score_lmetric":267108},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":998,"cache_hit":0,"new_prefill":87264,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":910,"cache_hit":0,"new_prefill":87264,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1056,"cache_hit":82432,"new_prefill":4832,"score_linear":-82432.0,"score_lmetric":0}],"chosen_score_linear":-82432.0,"chosen_score_lmetric":0,"t_first_token":281892.344544839,"t_first_token_unix":1779712757.3584883,"t_done":281894.328110254,"t_done_unix":1779712759.3420537},{"request_id":"1336620:1:1336620:423","session_id":"1336620","input_length":14666,"t_proxy_recv":281893.960309407,"t_decision_unix":1779712758.9742508,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":7680,"estimated_new_tokens":6986,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":900,"cache_hit":7680,"new_prefill":6986,"score_linear":47450.0,"score_lmetric":6986},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":14666,"score_linear":62618.0,"score_lmetric":14666},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":14666,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1251,"cache_hit":7680,"new_prefill":6986,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":218348,"ongoing_decode_tokens":0,"pending_prefill_tokens":1772,"num_requests":3,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":14666,"score_linear":218348.0,"score_lmetric":49314},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":998,"cache_hit":0,"new_prefill":14666,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":933,"cache_hit":7680,"new_prefill":6986,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":87264,"ongoing_decode_tokens":87264,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1056,"cache_hit":7680,"new_prefill":6986,"score_linear":79584.0,"score_lmetric":6986}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":281894.737622564,"t_first_token_unix":1779712759.751566,"t_done":281895.706057498,"t_done_unix":1779712760.7200005},{"request_id":"1259178:2:1336768:424","session_id":"1259178","input_length":20009,"t_proxy_recv":281894.425565986,"t_decision_unix":1779712759.4395068,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":14336,"affinity_cache_ratio":0.716477585086711,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":5673,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":20009,"score_linear":55130.0,"score_lmetric":20009},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":20009,"score_linear":62618.0,"score_lmetric":20009},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":20009,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":14666,"ongoing_decode_tokens":0,"pending_prefill_tokens":6986,"num_requests":1,"active_p_offloads":0,"cached_blocks":1251,"cache_hit":0,"new_prefill":20009,"score_linear":14666.0,"score_lmetric":26995},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":218348,"ongoing_decode_tokens":0,"pending_prefill_tokens":1772,"num_requests":3,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":20009,"score_linear":218348.0,"score_lmetric":65343},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":998,"cache_hit":0,"new_prefill":20009,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":933,"cache_hit":14336,"new_prefill":5673,"score_linear":-14336.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":20009,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-14336.0,"chosen_score_lmetric":0,"t_first_token":281896.48551208,"t_first_token_unix":1779712761.4994557,"t_done":281897.293948249,"t_done_unix":1779712762.3078914},{"request_id":"1257323:2:1329541:393","session_id":"1257323","input_length":90605,"t_proxy_recv":281869.217018968,"t_decision_unix":1779712734.2309604,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":90112,"affinity_cache_ratio":0.9945587991832681,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":90112,"estimated_new_tokens":493,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":888,"cache_hit":0,"new_prefill":90605,"score_linear":55130.0,"score_lmetric":90605},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":90605,"score_linear":62618.0,"score_lmetric":90605},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10847,"ongoing_decode_tokens":0,"pending_prefill_tokens":10847,"num_requests":1,"active_p_offloads":0,"cached_blocks":1101,"cache_hit":0,"new_prefill":90605,"score_linear":10847.0,"score_lmetric":101452},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":94823,"ongoing_decode_tokens":0,"pending_prefill_tokens":68199,"num_requests":2,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":90605,"score_linear":94823.0,"score_lmetric":317608},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":90112,"new_prefill":493,"score_linear":-90112.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":67446,"ongoing_decode_tokens":67446,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":90605,"score_linear":67446.0,"score_lmetric":181210},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":36880,"ongoing_decode_tokens":0,"pending_prefill_tokens":36880,"num_requests":1,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":90605,"score_linear":36880.0,"score_lmetric":127485},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":958,"cache_hit":0,"new_prefill":90605,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-90112.0,"chosen_score_lmetric":0,"t_first_token":281896.6871693,"t_first_token_unix":1779712761.7011127,"t_done":281897.626410712,"t_done_unix":1779712762.6403544},{"request_id":"1316848:13:1337519:427","session_id":"1316848","input_length":45005,"t_proxy_recv":281897.315173709,"t_decision_unix":1779712762.329115,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":44544,"affinity_cache_ratio":0.9897566937007,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":44544,"estimated_new_tokens":461,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":45005,"score_linear":55130.0,"score_lmetric":45005},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":45005,"score_linear":62618.0,"score_lmetric":45005},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":45005,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":44544,"new_prefill":461,"score_linear":-44544.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":218348,"ongoing_decode_tokens":218348,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":45005,"score_linear":218348.0,"score_lmetric":135015},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":49162,"ongoing_decode_tokens":12644,"pending_prefill_tokens":16550,"num_requests":2,"active_p_offloads":0,"cached_blocks":998,"cache_hit":0,"new_prefill":45005,"score_linear":49162.0,"score_lmetric":123110},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":944,"cache_hit":0,"new_prefill":45005,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":45005,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-44544.0,"chosen_score_lmetric":0,"t_first_token":281897.493752317,"t_first_token_unix":1779712762.5076957,"t_done":281897.792920824,"t_done_unix":1779712762.8068635},{"request_id":"1259178:3:1337653:429","session_id":"1259178","input_length":22398,"t_proxy_recv":281897.735297239,"t_decision_unix":1779712762.7492387,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":19968,"affinity_cache_ratio":0.8915081703723546,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":2430,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":22398,"score_linear":55130.0,"score_lmetric":22398},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":22398,"score_linear":62618.0,"score_lmetric":22398},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":22398,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":45005,"ongoing_decode_tokens":45005,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":22398,"score_linear":45005.0,"score_lmetric":22398},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":127743,"ongoing_decode_tokens":127743,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1075,"cache_hit":0,"new_prefill":22398,"score_linear":127743.0,"score_lmetric":44796},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":49162,"ongoing_decode_tokens":12644,"pending_prefill_tokens":16550,"num_requests":2,"active_p_offloads":0,"cached_blocks":998,"cache_hit":0,"new_prefill":22398,"score_linear":49162.0,"score_lmetric":77896},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":944,"cache_hit":19968,"new_prefill":2430,"score_linear":-19968.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95383,"ongoing_decode_tokens":0,"pending_prefill_tokens":8343,"num_requests":1,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":22398,"score_linear":95383.0,"score_lmetric":30741}],"chosen_score_linear":-19968.0,"chosen_score_lmetric":0,"t_first_token":281898.171594093,"t_first_token_unix":1779712763.185537,"t_done":281899.029290403,"t_done_unix":1779712764.0432339},{"request_id":"1309252:2:1334746:419","session_id":"1309252","input_length":45601,"t_proxy_recv":281887.549386595,"t_decision_unix":1779712752.563328,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":44544,"affinity_cache_ratio":0.9768206837569352,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":44544,"estimated_new_tokens":1057,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":45601,"score_linear":55130.0,"score_lmetric":45601},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":45601,"score_linear":62618.0,"score_lmetric":45601},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":45601,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":44664,"ongoing_decode_tokens":0,"pending_prefill_tokens":14968,"num_requests":1,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":0,"new_prefill":45601,"score_linear":44664.0,"score_lmetric":60569},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":44544,"new_prefill":1057,"score_linear":128203.0,"score_lmetric":3544},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":23689,"ongoing_decode_tokens":0,"pending_prefill_tokens":23689,"num_requests":1,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":45601,"score_linear":23689.0,"score_lmetric":69290},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":910,"cache_hit":0,"new_prefill":45601,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1056,"cache_hit":0,"new_prefill":45601,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":128203.0,"chosen_score_lmetric":3544,"t_first_token":281897.015282804,"t_first_token_unix":1779712762.029226,"t_done":281899.913589546,"t_done_unix":1779712764.927533},{"request_id":"1337035:1:1337035:425","session_id":"1337035","input_length":12644,"t_proxy_recv":281895.452934871,"t_decision_unix":1779712760.4668763,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":12644,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":12644,"score_linear":55130.0,"score_lmetric":12644},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":12644,"score_linear":62618.0,"score_lmetric":12644},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":12644,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":14666,"ongoing_decode_tokens":14666,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1251,"cache_hit":0,"new_prefill":12644,"score_linear":14666.0,"score_lmetric":12644},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":218348,"ongoing_decode_tokens":0,"pending_prefill_tokens":1772,"num_requests":3,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":12644,"score_linear":218348.0,"score_lmetric":43248},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":998,"cache_hit":0,"new_prefill":12644,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20009,"ongoing_decode_tokens":0,"pending_prefill_tokens":5673,"num_requests":1,"active_p_offloads":0,"cached_blocks":933,"cache_hit":0,"new_prefill":12644,"score_linear":20009.0,"score_lmetric":18317},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":12644,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281896.50979334,"t_first_token_unix":1779712761.5237367,"t_done":281900.10181811,"t_done_unix":1779712765.1157615},{"request_id":"1311753:2:1337278:426","session_id":"1311753","input_length":36518,"t_proxy_recv":281896.32211586,"t_decision_unix":1779712761.3360574,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":19968,"affinity_cache_ratio":0.5467988389287475,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":16550,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":36518,"score_linear":55130.0,"score_lmetric":36518},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":36518,"score_linear":62618.0,"score_lmetric":36518},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":36518,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":36518,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":218348,"ongoing_decode_tokens":0,"pending_prefill_tokens":1772,"num_requests":3,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":36518,"score_linear":218348.0,"score_lmetric":114870},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":12644,"ongoing_decode_tokens":0,"pending_prefill_tokens":12644,"num_requests":1,"active_p_offloads":0,"cached_blocks":998,"cache_hit":19968,"new_prefill":16550,"score_linear":-7324.0,"score_lmetric":29194},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20009,"ongoing_decode_tokens":0,"pending_prefill_tokens":5673,"num_requests":1,"active_p_offloads":0,"cached_blocks":933,"cache_hit":0,"new_prefill":36518,"score_linear":20009.0,"score_lmetric":42191},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":36518,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7324.0,"chosen_score_lmetric":29194,"t_first_token":281899.86576372,"t_first_token_unix":1779712764.8797073,"t_done":281900.122247769,"t_done_unix":1779712765.136191},{"request_id":"1338292:1:1338292:432","session_id":"1338292","input_length":9325,"t_proxy_recv":281899.940406532,"t_decision_unix":1779712764.954348,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9325,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":9325,"score_linear":55130.0,"score_lmetric":9325},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":9325,"score_linear":62618.0,"score_lmetric":9325},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":9325,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":9325,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82142,"ongoing_decode_tokens":82142,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":9325,"score_linear":82142.0,"score_lmetric":9325},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":49162,"ongoing_decode_tokens":49162,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":998,"cache_hit":0,"new_prefill":9325,"score_linear":49162.0,"score_lmetric":18650},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73073,"ongoing_decode_tokens":0,"pending_prefill_tokens":1393,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":9325,"score_linear":73073.0,"score_lmetric":10718},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95383,"ongoing_decode_tokens":0,"pending_prefill_tokens":8343,"num_requests":1,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":9325,"score_linear":95383.0,"score_lmetric":17668}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281900.64537853,"t_first_token_unix":1779712765.6593223,"t_done":281900.765077992,"t_done_unix":1779712765.7790215},{"request_id":"1269373:17:1338104:431","session_id":"1269373","input_length":73073,"t_proxy_recv":281899.400459715,"t_decision_unix":1779712764.414401,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":71680,"affinity_cache_ratio":0.9809368713478303,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71680,"estimated_new_tokens":1393,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":900,"cache_hit":7680,"new_prefill":65393,"score_linear":47450.0,"score_lmetric":65393},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":73073,"score_linear":62618.0,"score_lmetric":73073},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":73073,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":7680,"new_prefill":65393,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":127743,"ongoing_decode_tokens":127743,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1075,"cache_hit":0,"new_prefill":73073,"score_linear":127743.0,"score_lmetric":146146},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":49162,"ongoing_decode_tokens":12644,"pending_prefill_tokens":16550,"num_requests":2,"active_p_offloads":0,"cached_blocks":998,"cache_hit":0,"new_prefill":73073,"score_linear":49162.0,"score_lmetric":179246},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":71680,"new_prefill":1393,"score_linear":-71680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95383,"ongoing_decode_tokens":0,"pending_prefill_tokens":8343,"num_requests":1,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":7680,"new_prefill":65393,"score_linear":87703.0,"score_lmetric":73736}],"chosen_score_linear":-71680.0,"chosen_score_lmetric":0,"t_first_token":281900.044844295,"t_first_token_unix":1779712765.0587883,"t_done":281902.429576245,"t_done_unix":1779712767.4435198},{"request_id":"1317598:4:1332127:408","session_id":"1317598","input_length":82142,"t_proxy_recv":281878.001137734,"t_decision_unix":1779712743.0150793,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":81920,"affinity_cache_ratio":0.9972973631029194,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":81920,"estimated_new_tokens":222,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":891,"cache_hit":0,"new_prefill":82142,"score_linear":55130.0,"score_lmetric":82142},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":82142,"score_linear":62618.0,"score_lmetric":82142},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":21680,"ongoing_decode_tokens":0,"pending_prefill_tokens":21680,"num_requests":1,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":0,"new_prefill":82142,"score_linear":21680.0,"score_lmetric":103822},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":102211,"ongoing_decode_tokens":28977,"pending_prefill_tokens":6674,"num_requests":2,"active_p_offloads":0,"cached_blocks":1207,"cache_hit":0,"new_prefill":82142,"score_linear":102211.0,"score_lmetric":177632},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":81920,"new_prefill":222,"score_linear":8685.0,"score_lmetric":715},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":82142,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":71777,"ongoing_decode_tokens":0,"pending_prefill_tokens":2657,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":82142,"score_linear":71777.0,"score_lmetric":84799},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":12421,"ongoing_decode_tokens":0,"pending_prefill_tokens":12421,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":82142,"score_linear":12421.0,"score_lmetric":94563}],"chosen_score_linear":8685.0,"chosen_score_lmetric":715,"t_first_token":281897.012793051,"t_first_token_unix":1779712762.0267367,"t_done":281902.501017888,"t_done_unix":1779712767.5149615},{"request_id":"1317598:5:1337780:430","session_id":"1317598","input_length":83088,"t_proxy_recv":281902.522695647,"t_decision_unix":1779712767.536637,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":82432,"affinity_cache_ratio":0.99210475640285,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":82432,"estimated_new_tokens":656,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":55130,"pending_prefill_tokens":4712,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":83088,"score_linear":121282.0,"score_lmetric":175600},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":83088,"score_linear":62618.0,"score_lmetric":83088},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":83088,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":114821,"ongoing_decode_tokens":0,"pending_prefill_tokens":6277,"num_requests":3,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":83088,"score_linear":114821.0,"score_lmetric":268095},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1078,"cache_hit":82432,"new_prefill":656,"score_linear":-82432.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":83088,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":83088,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95383,"ongoing_decode_tokens":95383,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":83088,"score_linear":95383.0,"score_lmetric":83088}],"chosen_score_linear":-82432.0,"chosen_score_lmetric":0,"t_first_token":281902.949683916,"t_first_token_unix":1779712767.9636276,"t_done":281903.518126603,"t_done_unix":1779712768.53207},{"request_id":"1279412:20:1337591:428","session_id":"1279412","input_length":95383,"t_proxy_recv":281897.547278144,"t_decision_unix":1779712762.5612192,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":87040,"affinity_cache_ratio":0.912531583196167,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":87040,"estimated_new_tokens":8343,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":95383,"score_linear":55130.0,"score_lmetric":95383},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":95383,"score_linear":62618.0,"score_lmetric":95383},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":95383,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":45005,"ongoing_decode_tokens":45005,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":95383,"score_linear":45005.0,"score_lmetric":95383},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":218348,"ongoing_decode_tokens":218348,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":95383,"score_linear":218348.0,"score_lmetric":286149},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":49162,"ongoing_decode_tokens":12644,"pending_prefill_tokens":16550,"num_requests":2,"active_p_offloads":0,"cached_blocks":998,"cache_hit":0,"new_prefill":95383,"score_linear":49162.0,"score_lmetric":223866},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":944,"cache_hit":0,"new_prefill":95383,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":87040,"new_prefill":8343,"score_linear":-87040.0,"score_lmetric":0}],"chosen_score_linear":-87040.0,"chosen_score_lmetric":0,"t_first_token":281902.07584281,"t_first_token_unix":1779712767.089786,"t_done":281903.575804264,"t_done_unix":1779712768.5897477},{"request_id":"1279412:21:1339581:440","session_id":"1279412","input_length":95543,"t_proxy_recv":281904.508572066,"t_decision_unix":1779712769.5225132,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":95232,"affinity_cache_ratio":0.9967449211349864,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":95232,"estimated_new_tokens":311,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":121282,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":95543,"score_linear":121282.0,"score_lmetric":191086},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":95543,"score_linear":62618.0,"score_lmetric":95543},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":95543,"score_linear":7065.0,"score_lmetric":95543},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":114821,"ongoing_decode_tokens":0,"pending_prefill_tokens":6277,"num_requests":3,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":95543,"score_linear":114821.0,"score_lmetric":305460},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1079,"cache_hit":0,"new_prefill":95543,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":95543,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24462,"ongoing_decode_tokens":24462,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":95543,"score_linear":24462.0,"score_lmetric":95543},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1081,"cache_hit":95232,"new_prefill":311,"score_linear":-95232.0,"score_lmetric":0}],"chosen_score_linear":-95232.0,"chosen_score_lmetric":0,"t_first_token":281904.6513303,"t_first_token_unix":1779712769.6652737,"t_done":281906.96152712,"t_done_unix":1779712771.9754705},{"request_id":"1339977:1:1339977:442","session_id":"1339977","input_length":13816,"t_proxy_recv":281905.997349597,"t_decision_unix":1779712771.0112906,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":13816,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":121282,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":13816,"score_linear":121282.0,"score_lmetric":27632},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":13816,"score_linear":62618.0,"score_lmetric":13816},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":13816,"score_linear":7065.0,"score_lmetric":13816},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":114821,"ongoing_decode_tokens":0,"pending_prefill_tokens":6277,"num_requests":3,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":13816,"score_linear":114821.0,"score_lmetric":60279},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":45994,"ongoing_decode_tokens":45994,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1079,"cache_hit":0,"new_prefill":13816,"score_linear":45994.0,"score_lmetric":13816},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":13816,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24462,"ongoing_decode_tokens":24462,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":13816,"score_linear":24462.0,"score_lmetric":13816},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95543,"ongoing_decode_tokens":95543,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1081,"cache_hit":0,"new_prefill":13816,"score_linear":95543.0,"score_lmetric":13816}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281907.191501422,"t_first_token_unix":1779712772.205445,"t_done":281907.389855056,"t_done_unix":1779712772.403798},{"request_id":"1309252:3:1339771:441","session_id":"1309252","input_length":45994,"t_proxy_recv":281905.236422527,"t_decision_unix":1779712770.2503636,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":45568,"affinity_cache_ratio":0.9907379223376962,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":45568,"estimated_new_tokens":426,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":121282,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":45994,"score_linear":121282.0,"score_lmetric":91988},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":45994,"score_linear":62618.0,"score_lmetric":45994},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":45994,"score_linear":7065.0,"score_lmetric":45994},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":114821,"ongoing_decode_tokens":0,"pending_prefill_tokens":6277,"num_requests":3,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":45994,"score_linear":114821.0,"score_lmetric":156813},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1079,"cache_hit":45568,"new_prefill":426,"score_linear":-45568.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":45994,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24462,"ongoing_decode_tokens":24462,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":45994,"score_linear":24462.0,"score_lmetric":45994},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95543,"ongoing_decode_tokens":95543,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1081,"cache_hit":0,"new_prefill":45994,"score_linear":95543.0,"score_lmetric":45994}],"chosen_score_linear":-45568.0,"chosen_score_lmetric":0,"t_first_token":281905.403939243,"t_first_token_unix":1779712770.417883,"t_done":281908.840200159,"t_done_unix":1779712773.8541434},{"request_id":"1336620:2:1338440:434","session_id":"1336620","input_length":14918,"t_proxy_recv":281900.350092272,"t_decision_unix":1779712765.3640337,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":14336,"affinity_cache_ratio":0.960986727443357,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":582,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":900,"cache_hit":7680,"new_prefill":7238,"score_linear":47450.0,"score_lmetric":7238},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":14918,"score_linear":62618.0,"score_lmetric":14918},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":9325,"ongoing_decode_tokens":0,"pending_prefill_tokens":9325,"num_requests":1,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":14918,"score_linear":9325.0,"score_lmetric":24243},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":50215,"ongoing_decode_tokens":0,"pending_prefill_tokens":551,"num_requests":1,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":14336,"new_prefill":582,"score_linear":35879.0,"score_lmetric":1133},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82142,"ongoing_decode_tokens":82142,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":14918,"score_linear":82142.0,"score_lmetric":14918},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":14918,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73073,"ongoing_decode_tokens":73073,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":7680,"new_prefill":7238,"score_linear":65393.0,"score_lmetric":7238},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95383,"ongoing_decode_tokens":0,"pending_prefill_tokens":8343,"num_requests":1,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":7680,"new_prefill":7238,"score_linear":87703.0,"score_lmetric":15581}],"chosen_score_linear":35879.0,"chosen_score_lmetric":1133,"t_first_token":281906.8643034,"t_first_token_unix":1779712771.8782465,"t_done":281909.808842254,"t_done_unix":1779712774.8227856},{"request_id":"1341005:1:1341005:447","session_id":"1341005","input_length":5280,"t_proxy_recv":281909.465876408,"t_decision_unix":1779712774.4798148,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":5280,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":121282,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":5280,"score_linear":121282.0,"score_lmetric":10560},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":5280,"score_linear":62618.0,"score_lmetric":5280},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":5280,"score_linear":7065.0,"score_lmetric":5280},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":114821,"ongoing_decode_tokens":114821,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":5280,"score_linear":114821.0,"score_lmetric":15840},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":2279,"ongoing_decode_tokens":2279,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1080,"cache_hit":0,"new_prefill":5280,"score_linear":2279.0,"score_lmetric":5280},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1081,"cache_hit":0,"new_prefill":5280,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":45683,"ongoing_decode_tokens":45683,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":5280,"score_linear":45683.0,"score_lmetric":10560},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1082,"cache_hit":0,"new_prefill":5280,"score_linear":48381.0,"score_lmetric":53661}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281909.804300427,"t_first_token_unix":1779712774.8182437,"t_done":281910.166292127,"t_done_unix":1779712775.1802351},{"request_id":"1340903:1:1340903:446","session_id":"1340903","input_length":2279,"t_proxy_recv":281909.12618034,"t_decision_unix":1779712774.1401217,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":2279,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":121282,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":2279,"score_linear":121282.0,"score_lmetric":4558},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":2279,"score_linear":62618.0,"score_lmetric":2279},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":2279,"score_linear":7065.0,"score_lmetric":2279},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":114821,"ongoing_decode_tokens":114821,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":2279,"score_linear":114821.0,"score_lmetric":6837},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1080,"cache_hit":0,"new_prefill":2279,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1081,"cache_hit":0,"new_prefill":2279,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":45683,"ongoing_decode_tokens":24462,"pending_prefill_tokens":20197,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":2279,"score_linear":45683.0,"score_lmetric":44952},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1082,"cache_hit":0,"new_prefill":2279,"score_linear":48381.0,"score_lmetric":50660}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281909.273972059,"t_first_token_unix":1779712774.2879152,"t_done":281910.29563483,"t_done_unix":1779712775.3095782},{"request_id":"1259178:4:1339295:439","session_id":"1259178","input_length":24462,"t_proxy_recv":281903.412817469,"t_decision_unix":1779712768.426758,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":22528,"affinity_cache_ratio":0.9209385986427929,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":22528,"estimated_new_tokens":1934,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":55130,"pending_prefill_tokens":4712,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":24462,"score_linear":121282.0,"score_lmetric":58348},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":24462,"score_linear":62618.0,"score_lmetric":24462},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":24462,"score_linear":7065.0,"score_lmetric":24462},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":114821,"ongoing_decode_tokens":0,"pending_prefill_tokens":6277,"num_requests":3,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":24462,"score_linear":114821.0,"score_lmetric":92217},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83088,"ongoing_decode_tokens":83088,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1078,"cache_hit":0,"new_prefill":24462,"score_linear":83088.0,"score_lmetric":24462},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":24462,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":952,"cache_hit":22528,"new_prefill":1934,"score_linear":-22528.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95383,"ongoing_decode_tokens":95383,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":24462,"score_linear":95383.0,"score_lmetric":24462}],"chosen_score_linear":-22528.0,"chosen_score_lmetric":0,"t_first_token":281903.864115447,"t_first_token_unix":1779712768.8780587,"t_done":281910.677625211,"t_done_unix":1779712775.6915686},{"request_id":"1340290:1:1340290:445","session_id":"1340290","input_length":21221,"t_proxy_recv":281907.116654606,"t_decision_unix":1779712772.1305962,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":20197,"tie_break_used":false,"cache_hit":1024,"estimated_new_tokens":20197,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":121282,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":21221,"score_linear":121282.0,"score_lmetric":42442},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":21221,"score_linear":62618.0,"score_lmetric":21221},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":21221,"score_linear":7065.0,"score_lmetric":21221},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":114821,"ongoing_decode_tokens":65133,"pending_prefill_tokens":5144,"num_requests":3,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":21221,"score_linear":114821.0,"score_lmetric":79095},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":45994,"ongoing_decode_tokens":45994,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1079,"cache_hit":0,"new_prefill":21221,"score_linear":45994.0,"score_lmetric":21221},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13816,"ongoing_decode_tokens":0,"pending_prefill_tokens":13816,"num_requests":1,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":21221,"score_linear":13816.0,"score_lmetric":35037},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24462,"ongoing_decode_tokens":24462,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":952,"cache_hit":1024,"new_prefill":20197,"score_linear":23438.0,"score_lmetric":20197},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1082,"cache_hit":0,"new_prefill":21221,"score_linear":48381.0,"score_lmetric":69602}],"chosen_score_linear":23438.0,"chosen_score_lmetric":20197,"t_first_token":281909.348838383,"t_first_token_unix":1779712774.362782,"t_done":281911.498413505,"t_done_unix":1779712776.5123568},{"request_id":"1341912:1:1341912:449","session_id":"1341912","input_length":9909,"t_proxy_recv":281912.503337902,"t_decision_unix":1779712777.517279,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9909,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":121282,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":9909,"score_linear":121282.0,"score_lmetric":19818},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":9909,"score_linear":62618.0,"score_lmetric":9909},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":9909,"score_linear":7065.0,"score_lmetric":9909},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121274,"ongoing_decode_tokens":121274,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":9909,"score_linear":121274.0,"score_lmetric":29727},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1084,"cache_hit":0,"new_prefill":9909,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37527,"ongoing_decode_tokens":0,"pending_prefill_tokens":13975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1091,"cache_hit":0,"new_prefill":9909,"score_linear":37527.0,"score_lmetric":23884},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":997,"cache_hit":0,"new_prefill":9909,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1082,"cache_hit":0,"new_prefill":9909,"score_linear":48381.0,"score_lmetric":58290}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281913.265123795,"t_first_token_unix":1779712778.2790678,"t_done":281913.944590824,"t_done_unix":1779712778.9585335},{"request_id":"1340290:2:1342119:451","session_id":"1340290","input_length":21380,"t_proxy_recv":281913.164688734,"t_decision_unix":1779712778.17863,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":20992,"affinity_cache_ratio":0.9818521983161833,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":20992,"estimated_new_tokens":388,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":121282,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":21380,"score_linear":121282.0,"score_lmetric":42760},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":21380,"score_linear":62618.0,"score_lmetric":21380},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":21380,"score_linear":7065.0,"score_lmetric":21380},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121274,"ongoing_decode_tokens":121274,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":21380,"score_linear":121274.0,"score_lmetric":64140},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":19887,"ongoing_decode_tokens":0,"pending_prefill_tokens":19887,"num_requests":1,"active_p_offloads":0,"cached_blocks":1084,"cache_hit":0,"new_prefill":21380,"score_linear":19887.0,"score_lmetric":41267},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37527,"ongoing_decode_tokens":0,"pending_prefill_tokens":13975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1091,"cache_hit":0,"new_prefill":21380,"score_linear":37527.0,"score_lmetric":35355},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":9909,"ongoing_decode_tokens":0,"pending_prefill_tokens":9909,"num_requests":1,"active_p_offloads":0,"cached_blocks":997,"cache_hit":20992,"new_prefill":388,"score_linear":-11083.0,"score_lmetric":10297},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1082,"cache_hit":0,"new_prefill":21380,"score_linear":48381.0,"score_lmetric":69761}],"chosen_score_linear":-11083.0,"chosen_score_lmetric":10297,"t_first_token":281913.284297726,"t_first_token_unix":1779712778.2982414,"t_done":281914.092065882,"t_done_unix":1779712779.106009},{"request_id":"1341948:1:1341948:450","session_id":"1341948","input_length":19887,"t_proxy_recv":281912.631217341,"t_decision_unix":1779712777.6451588,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":19887,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":121282,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":19887,"score_linear":121282.0,"score_lmetric":39774},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":19887,"score_linear":62618.0,"score_lmetric":19887},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":19887,"score_linear":7065.0,"score_lmetric":19887},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121274,"ongoing_decode_tokens":121274,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":19887,"score_linear":121274.0,"score_lmetric":59661},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1084,"cache_hit":0,"new_prefill":19887,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37527,"ongoing_decode_tokens":0,"pending_prefill_tokens":13975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1091,"cache_hit":0,"new_prefill":19887,"score_linear":37527.0,"score_lmetric":33862},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":9909,"ongoing_decode_tokens":0,"pending_prefill_tokens":9909,"num_requests":1,"active_p_offloads":0,"cached_blocks":997,"cache_hit":0,"new_prefill":19887,"score_linear":9909.0,"score_lmetric":29796},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1082,"cache_hit":0,"new_prefill":19887,"score_linear":48381.0,"score_lmetric":68268}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281914.690785515,"t_first_token_unix":1779712779.7047293,"t_done":281914.943901438,"t_done_unix":1779712779.9578443},{"request_id":"1342327:1:1342327:452","session_id":"1342327","input_length":6180,"t_proxy_recv":281913.92555205,"t_decision_unix":1779712778.9394932,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":6180,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":6180,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":121282,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":6180,"score_linear":121282.0,"score_lmetric":12360},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":6180,"score_linear":62618.0,"score_lmetric":6180},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":6180,"score_linear":7065.0,"score_lmetric":6180},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121274,"ongoing_decode_tokens":121274,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":6180,"score_linear":121274.0,"score_lmetric":18540},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":19887,"ongoing_decode_tokens":0,"pending_prefill_tokens":19887,"num_requests":1,"active_p_offloads":0,"cached_blocks":1084,"cache_hit":0,"new_prefill":6180,"score_linear":19887.0,"score_lmetric":26067},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37527,"ongoing_decode_tokens":0,"pending_prefill_tokens":13975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1091,"cache_hit":0,"new_prefill":6180,"score_linear":37527.0,"score_lmetric":20155},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31289,"ongoing_decode_tokens":31289,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":997,"cache_hit":0,"new_prefill":6180,"score_linear":31289.0,"score_lmetric":12360},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1082,"cache_hit":0,"new_prefill":6180,"score_linear":48381.0,"score_lmetric":54561}],"chosen_score_linear":62618.0,"chosen_score_lmetric":6180,"t_first_token":281914.355045888,"t_first_token_unix":1779712779.3689897,"t_done":281914.945442085,"t_done_unix":1779712779.9593852},{"request_id":"1253743:5:1338406:433","session_id":"1253743","input_length":50215,"t_proxy_recv":281900.262436242,"t_decision_unix":1779712765.2763777,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9890271831126157,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":551,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":50215,"score_linear":55130.0,"score_lmetric":50215},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":4608,"new_prefill":45607,"score_linear":58010.0,"score_lmetric":45607},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":9325,"ongoing_decode_tokens":0,"pending_prefill_tokens":9325,"num_requests":1,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":50215,"score_linear":9325.0,"score_lmetric":59540},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":49664,"new_prefill":551,"score_linear":-49664.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82142,"ongoing_decode_tokens":82142,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":50215,"score_linear":82142.0,"score_lmetric":50215},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":50215,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73073,"ongoing_decode_tokens":73073,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":50215,"score_linear":73073.0,"score_lmetric":50215},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95383,"ongoing_decode_tokens":0,"pending_prefill_tokens":8343,"num_requests":1,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":50215,"score_linear":95383.0,"score_lmetric":58558}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":281906.863382535,"t_first_token_unix":1779712771.877326,"t_done":281915.030515621,"t_done_unix":1779712780.0444589},{"request_id":"1340903:2:1342632:453","session_id":"1340903","input_length":2129,"t_proxy_recv":281914.962529673,"t_decision_unix":1779712779.976471,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":2048,"affinity_cache_ratio":0.9619539689995303,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":2048,"estimated_new_tokens":81,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":121282,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":2129,"score_linear":121282.0,"score_lmetric":4258},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":955,"cache_hit":0,"new_prefill":2129,"score_linear":62618.0,"score_lmetric":2129},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":2129,"score_linear":7065.0,"score_lmetric":2129},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121274,"ongoing_decode_tokens":121274,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":2129,"score_linear":121274.0,"score_lmetric":6387},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":2048,"new_prefill":81,"score_linear":-2048.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37527,"ongoing_decode_tokens":0,"pending_prefill_tokens":13975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1091,"cache_hit":0,"new_prefill":2129,"score_linear":37527.0,"score_lmetric":16104},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":2129,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1082,"cache_hit":0,"new_prefill":2129,"score_linear":48381.0,"score_lmetric":50510}],"chosen_score_linear":-2048.0,"chosen_score_lmetric":0,"t_first_token":281914.976367895,"t_first_token_unix":1779712779.9903111,"t_done":281915.0503618,"t_done_unix":1779712780.064305},{"request_id":"1315567:3:1325346:372","session_id":"1315567","input_length":55130,"t_proxy_recv":281853.840475883,"t_decision_unix":1779712718.854417,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9937239252675494,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54784,"estimated_new_tokens":346,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":4312,"ongoing_decode_tokens":4312,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":878,"cache_hit":54784,"new_prefill":346,"score_linear":-50472.0,"score_lmetric":346},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":55130,"score_linear":62618.0,"score_lmetric":55130},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":55130,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":26994,"ongoing_decode_tokens":26994,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1068,"cache_hit":7680,"new_prefill":47450,"score_linear":19314.0,"score_lmetric":47450},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":55130,"score_linear":79663.0,"score_lmetric":55130},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":55130,"score_linear":33182.0,"score_lmetric":55130},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":780,"cache_hit":7680,"new_prefill":47450,"score_linear":114288.0,"score_lmetric":47450},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":939,"cache_hit":17408,"new_prefill":37722,"score_linear":-17408.0,"score_lmetric":0}],"chosen_score_linear":-50472.0,"chosen_score_lmetric":346,"t_first_token":281854.04710199,"t_first_token_unix":1779712719.0610454,"t_done":281915.44970932,"t_done_unix":1779712780.463653},{"request_id":"1340278:1:1340278:444","session_id":"1340278","input_length":48381,"t_proxy_recv":281907.095485201,"t_decision_unix":1779712772.1094265,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":48381,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":121282,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":48381,"score_linear":121282.0,"score_lmetric":96762},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":48381,"score_linear":62618.0,"score_lmetric":48381},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":48381,"score_linear":7065.0,"score_lmetric":48381},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":114821,"ongoing_decode_tokens":65133,"pending_prefill_tokens":5144,"num_requests":3,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":48381,"score_linear":114821.0,"score_lmetric":160575},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":45994,"ongoing_decode_tokens":45994,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1079,"cache_hit":0,"new_prefill":48381,"score_linear":45994.0,"score_lmetric":48381},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13816,"ongoing_decode_tokens":0,"pending_prefill_tokens":13816,"num_requests":1,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":48381,"score_linear":13816.0,"score_lmetric":62197},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24462,"ongoing_decode_tokens":24462,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":48381,"score_linear":24462.0,"score_lmetric":48381},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1082,"cache_hit":0,"new_prefill":48381,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281915.850337185,"t_first_token_unix":1779712780.864281,"t_done":281916.439058835,"t_done_unix":1779712781.4530022},{"request_id":"1316848:14:1338658:435","session_id":"1316848","input_length":49688,"t_proxy_recv":281901.17834435,"t_decision_unix":1779712766.1922855,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":44544,"affinity_cache_ratio":0.8964739977459346,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":44544,"estimated_new_tokens":5144,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":49688,"score_linear":55130.0,"score_lmetric":49688},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":49688,"score_linear":62618.0,"score_lmetric":49688},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":49688,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":65133,"ongoing_decode_tokens":0,"pending_prefill_tokens":1133,"num_requests":2,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":44544,"new_prefill":5144,"score_linear":20589.0,"score_lmetric":12554},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82142,"ongoing_decode_tokens":82142,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":49688,"score_linear":82142.0,"score_lmetric":49688},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":49688,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73073,"ongoing_decode_tokens":73073,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":49688,"score_linear":73073.0,"score_lmetric":49688},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95383,"ongoing_decode_tokens":0,"pending_prefill_tokens":8343,"num_requests":1,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":49688,"score_linear":95383.0,"score_lmetric":58031}],"chosen_score_linear":20589.0,"chosen_score_lmetric":12554,"t_first_token":281907.848968446,"t_first_token_unix":1779712772.8629117,"t_done":281916.545208553,"t_done_unix":1779712781.5591516},{"request_id":"1341005:2:1342961:456","session_id":"1341005","input_length":9360,"t_proxy_recv":281916.129304032,"t_decision_unix":1779712781.143245,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":5120,"affinity_cache_ratio":0.5470085470085471,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":5120,"estimated_new_tokens":4240,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":903,"cache_hit":0,"new_prefill":9360,"score_linear":66152.0,"score_lmetric":9360},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":955,"cache_hit":0,"new_prefill":9360,"score_linear":62618.0,"score_lmetric":9360},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":9360,"score_linear":7065.0,"score_lmetric":9360},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71059,"ongoing_decode_tokens":71059,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":9360,"score_linear":71059.0,"score_lmetric":18720},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":9360,"score_linear":65289.0,"score_lmetric":74649},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37527,"ongoing_decode_tokens":37527,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1091,"cache_hit":5120,"new_prefill":4240,"score_linear":32407.0,"score_lmetric":4240},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":59219,"ongoing_decode_tokens":0,"pending_prefill_tokens":59219,"num_requests":1,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":9360,"score_linear":59219.0,"score_lmetric":68579},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":48381,"ongoing_decode_tokens":48381,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1082,"cache_hit":0,"new_prefill":9360,"score_linear":48381.0,"score_lmetric":9360}],"chosen_score_linear":32407.0,"chosen_score_lmetric":4240,"t_first_token":281916.517517152,"t_first_token_unix":1779712781.5314605,"t_done":281916.950504753,"t_done_unix":1779712781.9644485},{"request_id":"1334223:2:1341796:448","session_id":"1334223","input_length":37527,"t_proxy_recv":281912.073042602,"t_decision_unix":1779712777.0869837,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":23552,"affinity_cache_ratio":0.6276014602819303,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":23552,"estimated_new_tokens":13975,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":121282,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":37527,"score_linear":121282.0,"score_lmetric":75054},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":37527,"score_linear":62618.0,"score_lmetric":37527},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":37527,"score_linear":7065.0,"score_lmetric":37527},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121274,"ongoing_decode_tokens":121274,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":37527,"score_linear":121274.0,"score_lmetric":112581},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1084,"cache_hit":0,"new_prefill":37527,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1091,"cache_hit":23552,"new_prefill":13975,"score_linear":-23552.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":997,"cache_hit":0,"new_prefill":37527,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1082,"cache_hit":0,"new_prefill":37527,"score_linear":48381.0,"score_lmetric":85908}],"chosen_score_linear":-23552.0,"chosen_score_lmetric":0,"t_first_token":281915.104513434,"t_first_token_unix":1779712780.1184568,"t_done":281917.454592293,"t_done_unix":1779712782.4685357},{"request_id":"1343191:1:1343191:457","session_id":"1343191","input_length":11912,"t_proxy_recv":281917.03493629,"t_decision_unix":1779712782.0488772,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":11912,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":903,"cache_hit":0,"new_prefill":11912,"score_linear":66152.0,"score_lmetric":11912},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":955,"cache_hit":0,"new_prefill":11912,"score_linear":62618.0,"score_lmetric":11912},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":11912,"score_linear":7065.0,"score_lmetric":11912},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1277,"cache_hit":0,"new_prefill":11912,"score_linear":21371.0,"score_lmetric":11912},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":11912,"score_linear":65289.0,"score_lmetric":77201},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37527,"ongoing_decode_tokens":37527,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1099,"cache_hit":0,"new_prefill":11912,"score_linear":37527.0,"score_lmetric":11912},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":59219,"ongoing_decode_tokens":0,"pending_prefill_tokens":59219,"num_requests":1,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":11912,"score_linear":59219.0,"score_lmetric":71131},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1176,"cache_hit":0,"new_prefill":11912,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281917.990946918,"t_first_token_unix":1779712783.0048904,"t_done":281918.044525,"t_done_unix":1779712783.058468},{"request_id":"1338292:2:1343599:460","session_id":"1338292","input_length":14103,"t_proxy_recv":281918.244193539,"t_decision_unix":1779712783.258135,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":9216,"affinity_cache_ratio":0.6534779834077856,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9216,"estimated_new_tokens":4887,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":903,"cache_hit":0,"new_prefill":14103,"score_linear":66152.0,"score_lmetric":14103},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":955,"cache_hit":0,"new_prefill":14103,"score_linear":62618.0,"score_lmetric":14103},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":9216,"new_prefill":4887,"score_linear":-2151.0,"score_lmetric":4887},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1277,"cache_hit":0,"new_prefill":14103,"score_linear":21371.0,"score_lmetric":14103},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":14103,"score_linear":65289.0,"score_lmetric":79392},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":21402,"ongoing_decode_tokens":0,"pending_prefill_tokens":21402,"num_requests":1,"active_p_offloads":0,"cached_blocks":1126,"cache_hit":0,"new_prefill":14103,"score_linear":21402.0,"score_lmetric":35505},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":69451,"ongoing_decode_tokens":0,"pending_prefill_tokens":59723,"num_requests":2,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":14103,"score_linear":69451.0,"score_lmetric":147652},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":0,"new_prefill":14103,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-2151.0,"chosen_score_lmetric":4887,"t_first_token":281918.80933503,"t_first_token_unix":1779712783.823279,"t_done":281918.894460291,"t_done_unix":1779712783.9084036},{"request_id":"1343312:1:1343312:459","session_id":"1343312","input_length":21402,"t_proxy_recv":281917.398796303,"t_decision_unix":1779712782.4127374,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":21402,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":21402,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":903,"cache_hit":0,"new_prefill":21402,"score_linear":66152.0,"score_lmetric":21402},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":955,"cache_hit":0,"new_prefill":21402,"score_linear":62618.0,"score_lmetric":21402},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":21402,"score_linear":7065.0,"score_lmetric":21402},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1277,"cache_hit":0,"new_prefill":21402,"score_linear":21371.0,"score_lmetric":21402},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":21402,"score_linear":65289.0,"score_lmetric":86691},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37527,"ongoing_decode_tokens":37527,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1099,"cache_hit":0,"new_prefill":21402,"score_linear":37527.0,"score_lmetric":21402},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":69451,"ongoing_decode_tokens":0,"pending_prefill_tokens":59723,"num_requests":2,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":21402,"score_linear":69451.0,"score_lmetric":162250},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":11912,"ongoing_decode_tokens":0,"pending_prefill_tokens":11912,"num_requests":1,"active_p_offloads":0,"cached_blocks":1176,"cache_hit":0,"new_prefill":21402,"score_linear":11912.0,"score_lmetric":33314}],"chosen_score_linear":37527.0,"chosen_score_lmetric":21402,"t_first_token":281919.696259698,"t_first_token_unix":1779712784.710204,"t_done":281919.752105053,"t_done_unix":1779712784.7660482},{"request_id":"1342327:2:1343857:461","session_id":"1342327","input_length":9820,"t_proxy_recv":281919.06270993,"t_decision_unix":1779712784.0766504,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":6144,"affinity_cache_ratio":0.6256619144602852,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":6144,"estimated_new_tokens":3676,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":903,"cache_hit":0,"new_prefill":9820,"score_linear":66152.0,"score_lmetric":9820},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":955,"cache_hit":6144,"new_prefill":3676,"score_linear":56474.0,"score_lmetric":3676},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1224,"cache_hit":0,"new_prefill":9820,"score_linear":7065.0,"score_lmetric":9820},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1277,"cache_hit":0,"new_prefill":9820,"score_linear":21371.0,"score_lmetric":9820},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":9820,"score_linear":65289.0,"score_lmetric":75109},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":21402,"ongoing_decode_tokens":0,"pending_prefill_tokens":21402,"num_requests":1,"active_p_offloads":0,"cached_blocks":1126,"cache_hit":0,"new_prefill":9820,"score_linear":21402.0,"score_lmetric":31222},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":69451,"ongoing_decode_tokens":0,"pending_prefill_tokens":59723,"num_requests":2,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":9820,"score_linear":69451.0,"score_lmetric":139086},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":0,"new_prefill":9820,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":56474.0,"chosen_score_lmetric":3676,"t_first_token":281919.435036711,"t_first_token_unix":1779712784.4489808,"t_done":281919.88119643,"t_done_unix":1779712784.8951402},{"request_id":"1314357:3:1338999:437","session_id":"1314357","input_length":66152,"t_proxy_recv":281902.388309055,"t_decision_unix":1779712767.4022503,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":61440,"affinity_cache_ratio":0.9287701052122385,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":61440,"estimated_new_tokens":4712,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":900,"cache_hit":61440,"new_prefill":4712,"score_linear":-6310.0,"score_lmetric":4712},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":66152,"score_linear":62618.0,"score_lmetric":66152},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":66152,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":114821,"ongoing_decode_tokens":0,"pending_prefill_tokens":6277,"num_requests":3,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":66152,"score_linear":114821.0,"score_lmetric":217287},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82142,"ongoing_decode_tokens":82142,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":66152,"score_linear":82142.0,"score_lmetric":66152},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":66152,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73073,"ongoing_decode_tokens":73073,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":66152,"score_linear":73073.0,"score_lmetric":66152},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95383,"ongoing_decode_tokens":95383,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":66152,"score_linear":95383.0,"score_lmetric":66152}],"chosen_score_linear":-6310.0,"chosen_score_lmetric":4712,"t_first_token":281904.447518184,"t_first_token_unix":1779712769.4614625,"t_done":281920.042917818,"t_done_unix":1779712785.0568612},{"request_id":"1316848:15:1344261:463","session_id":"1316848","input_length":50447,"t_proxy_recv":281920.472187387,"t_decision_unix":1779712785.4861286,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":50176,"affinity_cache_ratio":0.9946280254524551,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":50176,"estimated_new_tokens":271,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":914,"cache_hit":0,"new_prefill":50447,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":962,"cache_hit":0,"new_prefill":50447,"score_linear":62618.0,"score_lmetric":50447},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1224,"cache_hit":0,"new_prefill":50447,"score_linear":7065.0,"score_lmetric":50447},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1277,"cache_hit":50176,"new_prefill":271,"score_linear":-28805.0,"score_lmetric":271},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":50447,"score_linear":65289.0,"score_lmetric":115736},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1167,"cache_hit":0,"new_prefill":50447,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":69451,"ongoing_decode_tokens":0,"pending_prefill_tokens":59723,"num_requests":2,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":50447,"score_linear":69451.0,"score_lmetric":220340},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95781,"ongoing_decode_tokens":0,"pending_prefill_tokens":37,"num_requests":1,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":0,"new_prefill":50447,"score_linear":95781.0,"score_lmetric":50484}],"chosen_score_linear":-28805.0,"chosen_score_lmetric":271,"t_first_token":281920.648240131,"t_first_token_unix":1779712785.6621838,"t_done":281921.082427173,"t_done_unix":1779712786.0963707},{"request_id":"1279412:22:1344210:462","session_id":"1279412","input_length":95781,"t_proxy_recv":281920.333016064,"t_decision_unix":1779712785.3469577,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":95744,"affinity_cache_ratio":0.9996137020912289,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":95744,"estimated_new_tokens":37,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":914,"cache_hit":0,"new_prefill":95781,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":962,"cache_hit":0,"new_prefill":95781,"score_linear":62618.0,"score_lmetric":95781},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1224,"cache_hit":0,"new_prefill":95781,"score_linear":7065.0,"score_lmetric":95781},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1277,"cache_hit":0,"new_prefill":95781,"score_linear":21371.0,"score_lmetric":95781},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":95781,"score_linear":65289.0,"score_lmetric":161070},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1167,"cache_hit":0,"new_prefill":95781,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":69451,"ongoing_decode_tokens":0,"pending_prefill_tokens":59723,"num_requests":2,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":95781,"score_linear":69451.0,"score_lmetric":311008},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":95744,"new_prefill":37,"score_linear":-95744.0,"score_lmetric":0}],"chosen_score_linear":-95744.0,"chosen_score_lmetric":0,"t_first_token":281920.477463541,"t_first_token_unix":1779712785.4914067,"t_done":281922.596948322,"t_done_unix":1779712787.6108947},{"request_id":"1342327:3:1344788:466","session_id":"1342327","input_length":12398,"t_proxy_recv":281922.304258575,"t_decision_unix":1779712787.3181999,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":9728,"affinity_cache_ratio":0.78464268430392,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9728,"estimated_new_tokens":2670,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":32816,"ongoing_decode_tokens":0,"pending_prefill_tokens":32816,"num_requests":1,"active_p_offloads":0,"cached_blocks":914,"cache_hit":0,"new_prefill":12398,"score_linear":32816.0,"score_lmetric":45214},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":962,"cache_hit":9728,"new_prefill":2670,"score_linear":52890.0,"score_lmetric":2670},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1224,"cache_hit":0,"new_prefill":12398,"score_linear":7065.0,"score_lmetric":12398},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1277,"cache_hit":0,"new_prefill":12398,"score_linear":21371.0,"score_lmetric":12398},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":12398,"score_linear":65289.0,"score_lmetric":77687},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1167,"cache_hit":0,"new_prefill":12398,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":69451,"ongoing_decode_tokens":0,"pending_prefill_tokens":59723,"num_requests":2,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":12398,"score_linear":69451.0,"score_lmetric":144242},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95781,"ongoing_decode_tokens":95781,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":0,"new_prefill":12398,"score_linear":95781.0,"score_lmetric":12398}],"chosen_score_linear":52890.0,"chosen_score_lmetric":2670,"t_first_token":281922.636420395,"t_first_token_unix":1779712787.650364,"t_done":281922.911651003,"t_done_unix":1779712787.9255943},{"request_id":"1345921:1:1345921:469","session_id":"1345921","input_length":505,"t_proxy_recv":281926.16406901,"t_decision_unix":1779712791.1780102,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":505,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":32816,"ongoing_decode_tokens":0,"pending_prefill_tokens":32816,"num_requests":1,"active_p_offloads":0,"cached_blocks":914,"cache_hit":0,"new_prefill":505,"score_linear":32816.0,"score_lmetric":33321},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":967,"cache_hit":0,"new_prefill":505,"score_linear":62618.0,"score_lmetric":505},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1224,"cache_hit":0,"new_prefill":505,"score_linear":7065.0,"score_lmetric":505},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":80186,"ongoing_decode_tokens":21371,"pending_prefill_tokens":8639,"num_requests":2,"active_p_offloads":0,"cached_blocks":1277,"cache_hit":0,"new_prefill":505,"score_linear":80186.0,"score_lmetric":18288},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":505,"score_linear":65289.0,"score_lmetric":65794},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1167,"cache_hit":0,"new_prefill":505,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":143397,"ongoing_decode_tokens":0,"pending_prefill_tokens":60453,"num_requests":3,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":505,"score_linear":143397.0,"score_lmetric":182874},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":0,"new_prefill":505,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281926.216730802,"t_first_token_unix":1779712791.2306745,"t_done":281926.544847243,"t_done_unix":1779712791.5587904},{"request_id":"1344773:1:1344773:465","session_id":"1344773","input_length":32816,"t_proxy_recv":281922.26617061,"t_decision_unix":1779712787.2801116,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":32816,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":914,"cache_hit":0,"new_prefill":32816,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":962,"cache_hit":0,"new_prefill":32816,"score_linear":62618.0,"score_lmetric":32816},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1224,"cache_hit":0,"new_prefill":32816,"score_linear":7065.0,"score_lmetric":32816},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1277,"cache_hit":0,"new_prefill":32816,"score_linear":21371.0,"score_lmetric":32816},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":32816,"score_linear":65289.0,"score_lmetric":98105},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1167,"cache_hit":0,"new_prefill":32816,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":69451,"ongoing_decode_tokens":0,"pending_prefill_tokens":59723,"num_requests":2,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":32816,"score_linear":69451.0,"score_lmetric":185078},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95781,"ongoing_decode_tokens":95781,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":0,"new_prefill":32816,"score_linear":95781.0,"score_lmetric":32816}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281926.833537442,"t_first_token_unix":1779712791.847481,"t_done":281926.975726348,"t_done_unix":1779712791.9896696},{"request_id":"1342327:4:1346218:471","session_id":"1342327","input_length":14842,"t_proxy_recv":281927.265318097,"t_decision_unix":1779712792.2792592,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":12288,"affinity_cache_ratio":0.8279207653954993,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":12288,"estimated_new_tokens":2554,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":978,"cache_hit":0,"new_prefill":14842,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":967,"cache_hit":12288,"new_prefill":2554,"score_linear":50330.0,"score_lmetric":2554},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1224,"cache_hit":0,"new_prefill":14842,"score_linear":7065.0,"score_lmetric":14842},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":80186,"ongoing_decode_tokens":21371,"pending_prefill_tokens":8639,"num_requests":2,"active_p_offloads":0,"cached_blocks":1277,"cache_hit":0,"new_prefill":14842,"score_linear":80186.0,"score_lmetric":46962},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":14842,"score_linear":65289.0,"score_lmetric":80131},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1167,"cache_hit":0,"new_prefill":14842,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":143397,"ongoing_decode_tokens":0,"pending_prefill_tokens":60453,"num_requests":3,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":14842,"score_linear":143397.0,"score_lmetric":225885},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":14842,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":50330.0,"chosen_score_lmetric":2554,"t_first_token":281927.624527059,"t_first_token_unix":1779712792.638471,"t_done":281927.719993766,"t_done_unix":1779712792.7339368},{"request_id":"1344773:2:1346419:473","session_id":"1344773","input_length":33008,"t_proxy_recv":281928.033651014,"t_decision_unix":1779712793.0475922,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":32768,"affinity_cache_ratio":0.9927290353853612,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":32768,"estimated_new_tokens":240,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":978,"cache_hit":32768,"new_prefill":240,"score_linear":-32768.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":33008,"score_linear":62618.0,"score_lmetric":33008},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1224,"cache_hit":0,"new_prefill":33008,"score_linear":7065.0,"score_lmetric":33008},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":80186,"ongoing_decode_tokens":21371,"pending_prefill_tokens":8639,"num_requests":2,"active_p_offloads":0,"cached_blocks":1277,"cache_hit":0,"new_prefill":33008,"score_linear":80186.0,"score_lmetric":83294},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":33008,"score_linear":65289.0,"score_lmetric":98297},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37019,"ongoing_decode_tokens":0,"pending_prefill_tokens":2715,"num_requests":1,"active_p_offloads":0,"cached_blocks":1167,"cache_hit":0,"new_prefill":33008,"score_linear":37019.0,"score_lmetric":35723},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":143397,"ongoing_decode_tokens":69451,"pending_prefill_tokens":730,"num_requests":3,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":33008,"score_linear":143397.0,"score_lmetric":101214},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":33008,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-32768.0,"chosen_score_lmetric":0,"t_first_token":281928.157876115,"t_first_token_unix":1779712793.1718197,"t_done":281928.334573198,"t_done_unix":1779712793.348516},{"request_id":"1316848:16:1345678:468","session_id":"1316848","input_length":58815,"t_proxy_recv":281925.28802566,"t_decision_unix":1779712790.3019671,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":50176,"affinity_cache_ratio":0.8531157017767577,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":50176,"estimated_new_tokens":8639,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":32816,"ongoing_decode_tokens":0,"pending_prefill_tokens":32816,"num_requests":1,"active_p_offloads":0,"cached_blocks":914,"cache_hit":0,"new_prefill":58815,"score_linear":32816.0,"score_lmetric":91631},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":967,"cache_hit":0,"new_prefill":58815,"score_linear":62618.0,"score_lmetric":58815},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1224,"cache_hit":0,"new_prefill":58815,"score_linear":7065.0,"score_lmetric":58815},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1277,"cache_hit":50176,"new_prefill":8639,"score_linear":-28805.0,"score_lmetric":8639},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":58815,"score_linear":65289.0,"score_lmetric":124104},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1167,"cache_hit":0,"new_prefill":58815,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":143397,"ongoing_decode_tokens":0,"pending_prefill_tokens":60453,"num_requests":3,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":58815,"score_linear":143397.0,"score_lmetric":357804},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":0,"new_prefill":58815,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-28805.0,"chosen_score_lmetric":8639,"t_first_token":281928.311941749,"t_first_token_unix":1779712793.3258853,"t_done":281928.477206422,"t_done_unix":1779712793.4911494},{"request_id":"1268861:6:1346340:472","session_id":"1268861","input_length":37019,"t_proxy_recv":281927.706474832,"t_decision_unix":1779712792.720416,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9266592830708555,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":34304,"estimated_new_tokens":2715,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":978,"cache_hit":0,"new_prefill":37019,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77460,"ongoing_decode_tokens":77460,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":967,"cache_hit":0,"new_prefill":37019,"score_linear":77460.0,"score_lmetric":74038},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1224,"cache_hit":0,"new_prefill":37019,"score_linear":7065.0,"score_lmetric":37019},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":80186,"ongoing_decode_tokens":21371,"pending_prefill_tokens":8639,"num_requests":2,"active_p_offloads":0,"cached_blocks":1277,"cache_hit":0,"new_prefill":37019,"score_linear":80186.0,"score_lmetric":91316},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":21504,"new_prefill":15515,"score_linear":43785.0,"score_lmetric":80804},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1167,"cache_hit":34304,"new_prefill":2715,"score_linear":-34304.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":143397,"ongoing_decode_tokens":0,"pending_prefill_tokens":60453,"num_requests":3,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":37019,"score_linear":143397.0,"score_lmetric":292416},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":37019,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-34304.0,"chosen_score_lmetric":0,"t_first_token":281928.369889493,"t_first_token_unix":1779712793.3838327,"t_done":281930.217936714,"t_done_unix":1779712795.2318797},{"request_id":"1342634:1:1342634:454","session_id":"1342634","input_length":59219,"t_proxy_recv":281914.996386926,"t_decision_unix":1779712780.0103283,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":59219,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":121282,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":59219,"score_linear":121282.0,"score_lmetric":118438},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":955,"cache_hit":0,"new_prefill":59219,"score_linear":62618.0,"score_lmetric":59219},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":59219,"score_linear":7065.0,"score_lmetric":59219},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121274,"ongoing_decode_tokens":121274,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":59219,"score_linear":121274.0,"score_lmetric":177657},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":2129,"ongoing_decode_tokens":2129,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":59219,"score_linear":2129.0,"score_lmetric":59219},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37527,"ongoing_decode_tokens":0,"pending_prefill_tokens":13975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1091,"cache_hit":0,"new_prefill":59219,"score_linear":37527.0,"score_lmetric":73194},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":59219,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1082,"cache_hit":0,"new_prefill":59219,"score_linear":48381.0,"score_lmetric":107600}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281927.7447011,"t_first_token_unix":1779712792.7586446,"t_done":281930.311417345,"t_done_unix":1779712795.3253603},{"request_id":"1345921:2:1347089:474","session_id":"1345921","input_length":773,"t_proxy_recv":281930.350040537,"t_decision_unix":1779712795.3639817,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":512,"affinity_cache_ratio":0.6623544631306598,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":512,"estimated_new_tokens":261,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":978,"cache_hit":0,"new_prefill":773,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":773,"score_linear":62618.0,"score_lmetric":773},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1224,"cache_hit":0,"new_prefill":773,"score_linear":7065.0,"score_lmetric":773},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":773,"score_linear":21371.0,"score_lmetric":773},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":773,"score_linear":65289.0,"score_lmetric":66062},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1172,"cache_hit":0,"new_prefill":773,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":84178,"ongoing_decode_tokens":84178,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1127,"cache_hit":0,"new_prefill":773,"score_linear":84178.0,"score_lmetric":1546},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":512,"new_prefill":261,"score_linear":-512.0,"score_lmetric":0}],"chosen_score_linear":-512.0,"chosen_score_lmetric":0,"t_first_token":281930.383867942,"t_first_token_unix":1779712795.397811,"t_done":281930.637142159,"t_done_unix":1779712795.6510854},{"request_id":"1341912:2:1343220:458","session_id":"1341912","input_length":10232,"t_proxy_recv":281917.111373441,"t_decision_unix":1779712782.1253145,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":9728,"affinity_cache_ratio":0.9507427677873339,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9728,"estimated_new_tokens":504,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":903,"cache_hit":0,"new_prefill":10232,"score_linear":66152.0,"score_lmetric":10232},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":955,"cache_hit":0,"new_prefill":10232,"score_linear":62618.0,"score_lmetric":10232},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":10232,"score_linear":7065.0,"score_lmetric":10232},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1277,"cache_hit":0,"new_prefill":10232,"score_linear":21371.0,"score_lmetric":10232},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":10232,"score_linear":65289.0,"score_lmetric":75521},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37527,"ongoing_decode_tokens":37527,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1099,"cache_hit":0,"new_prefill":10232,"score_linear":37527.0,"score_lmetric":10232},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":59219,"ongoing_decode_tokens":0,"pending_prefill_tokens":59219,"num_requests":1,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":9728,"new_prefill":504,"score_linear":49491.0,"score_lmetric":59723},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":11912,"ongoing_decode_tokens":0,"pending_prefill_tokens":11912,"num_requests":1,"active_p_offloads":0,"cached_blocks":1176,"cache_hit":0,"new_prefill":10232,"score_linear":11912.0,"score_lmetric":22144}],"chosen_score_linear":49491.0,"chosen_score_lmetric":59723,"t_first_token":281927.746427987,"t_first_token_unix":1779712792.7603712,"t_done":281930.701376803,"t_done_unix":1779712795.7153196},{"request_id":"1339070:1:1339070:438","session_id":"1339070","input_length":7065,"t_proxy_recv":281902.585673863,"t_decision_unix":1779712767.5996156,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7065,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":55130,"pending_prefill_tokens":4712,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":7065,"score_linear":121282.0,"score_lmetric":23554},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":7065,"score_linear":62618.0,"score_lmetric":7065},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":7065,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":114821,"ongoing_decode_tokens":0,"pending_prefill_tokens":6277,"num_requests":3,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":7065,"score_linear":114821.0,"score_lmetric":40026},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83088,"ongoing_decode_tokens":0,"pending_prefill_tokens":656,"num_requests":1,"active_p_offloads":0,"cached_blocks":1078,"cache_hit":0,"new_prefill":7065,"score_linear":83088.0,"score_lmetric":7721},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":7065,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":7065,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95383,"ongoing_decode_tokens":95383,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":7065,"score_linear":95383.0,"score_lmetric":7065}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281903.063771838,"t_first_token_unix":1779712768.0777152,"t_done":281931.000774268,"t_done_unix":1779712796.0147176},{"request_id":"1269373:18:1345570:467","session_id":"1269373","input_length":73946,"t_proxy_recv":281924.837197698,"t_decision_unix":1779712789.851139,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":73216,"affinity_cache_ratio":0.9901279311930328,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73216,"estimated_new_tokens":730,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":32816,"ongoing_decode_tokens":0,"pending_prefill_tokens":32816,"num_requests":1,"active_p_offloads":0,"cached_blocks":914,"cache_hit":7680,"new_prefill":66266,"score_linear":25136.0,"score_lmetric":99082},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":967,"cache_hit":0,"new_prefill":73946,"score_linear":62618.0,"score_lmetric":73946},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1224,"cache_hit":0,"new_prefill":73946,"score_linear":7065.0,"score_lmetric":73946},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1277,"cache_hit":7680,"new_prefill":66266,"score_linear":13691.0,"score_lmetric":66266},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":73946,"score_linear":65289.0,"score_lmetric":139235},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1167,"cache_hit":0,"new_prefill":73946,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":69451,"ongoing_decode_tokens":0,"pending_prefill_tokens":59723,"num_requests":2,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":73216,"new_prefill":730,"score_linear":-3765.0,"score_lmetric":120906},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":7680,"new_prefill":66266,"score_linear":-7680.0,"score_lmetric":0}],"chosen_score_linear":-3765.0,"chosen_score_lmetric":120906,"t_first_token":281928.104471808,"t_first_token_unix":1779712793.1184154,"t_done":281932.649811771,"t_done_unix":1779712797.6637607},{"request_id":"1341948:2:1347296:475","session_id":"1341948","input_length":25661,"t_proxy_recv":281931.068327596,"t_decision_unix":1779712796.082269,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":19456,"affinity_cache_ratio":0.7581933673668212,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":19456,"estimated_new_tokens":6205,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":978,"cache_hit":0,"new_prefill":25661,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":25661,"score_linear":62618.0,"score_lmetric":25661},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1239,"cache_hit":0,"new_prefill":25661,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":25661,"score_linear":21371.0,"score_lmetric":25661},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":19456,"new_prefill":6205,"score_linear":45833.0,"score_lmetric":71494},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1172,"cache_hit":0,"new_prefill":25661,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73946,"ongoing_decode_tokens":73946,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1128,"cache_hit":0,"new_prefill":25661,"score_linear":73946.0,"score_lmetric":25661},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":25661,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":45833.0,"chosen_score_lmetric":71494,"t_first_token":281932.189950646,"t_first_token_unix":1779712797.2038968,"t_done":281932.698686722,"t_done_unix":1779712797.7126298},{"request_id":"1347763:1:1347763:476","session_id":"1347763","input_length":5465,"t_proxy_recv":281932.674697439,"t_decision_unix":1779712797.6886387,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":978,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":5465,"score_linear":62618.0,"score_lmetric":5465},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1239,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":5465,"score_linear":21371.0,"score_lmetric":5465},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90950,"ongoing_decode_tokens":90950,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":5465,"score_linear":90950.0,"score_lmetric":10930},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1172,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281933.029146807,"t_first_token_unix":1779712798.0430903,"t_done":281933.153804116,"t_done_unix":1779712798.1677477},{"request_id":"1342921:1:1342921:455","session_id":"1342921","input_length":65289,"t_proxy_recv":281916.040288211,"t_decision_unix":1779712781.0542293,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":65289,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":903,"cache_hit":0,"new_prefill":65289,"score_linear":66152.0,"score_lmetric":65289},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":955,"cache_hit":0,"new_prefill":65289,"score_linear":62618.0,"score_lmetric":65289},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":65289,"score_linear":7065.0,"score_lmetric":65289},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71059,"ongoing_decode_tokens":71059,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":65289,"score_linear":71059.0,"score_lmetric":130578},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1122,"cache_hit":0,"new_prefill":65289,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37527,"ongoing_decode_tokens":37527,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1091,"cache_hit":0,"new_prefill":65289,"score_linear":37527.0,"score_lmetric":65289},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":59219,"ongoing_decode_tokens":0,"pending_prefill_tokens":59219,"num_requests":1,"active_p_offloads":0,"cached_blocks":1016,"cache_hit":0,"new_prefill":65289,"score_linear":59219.0,"score_lmetric":124508},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":48381,"ongoing_decode_tokens":48381,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1082,"cache_hit":0,"new_prefill":65289,"score_linear":48381.0,"score_lmetric":65289}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281931.101507682,"t_first_token_unix":1779712796.115451,"t_done":281935.605463661,"t_done_unix":1779712800.619407},{"request_id":"1345921:3:1348784:483","session_id":"1345921","input_length":1397,"t_proxy_recv":281936.349176717,"t_decision_unix":1779712801.363118,"policy":"unified","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":4,"affinity_cache_hit":512,"affinity_cache_ratio":0.36649964209019326,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1397,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":12701,"ongoing_decode_tokens":0,"pending_prefill_tokens":12701,"num_requests":1,"active_p_offloads":0,"cached_blocks":978,"cache_hit":0,"new_prefill":1397,"score_linear":12701.0,"score_lmetric":14098},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":1397,"score_linear":62618.0,"score_lmetric":1397},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1239,"cache_hit":0,"new_prefill":1397,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":1397,"score_linear":21371.0,"score_lmetric":1397},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1262,"cache_hit":0,"new_prefill":1397,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":80838,"ongoing_decode_tokens":41202,"pending_prefill_tokens":3284,"num_requests":2,"active_p_offloads":0,"cached_blocks":1182,"cache_hit":0,"new_prefill":1397,"score_linear":80838.0,"score_lmetric":9362},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":17217,"ongoing_decode_tokens":0,"pending_prefill_tokens":17217,"num_requests":1,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":1397,"score_linear":17217.0,"score_lmetric":18614},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":512,"new_prefill":885,"score_linear":95514.0,"score_lmetric":885}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281936.447107764,"t_first_token_unix":1779712801.461051,"t_done":281936.566058433,"t_done_unix":1779712801.580002},{"request_id":"1311753:3:1348389:480","session_id":"1311753","input_length":39636,"t_proxy_recv":281935.156150916,"t_decision_unix":1779712800.170092,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":36352,"affinity_cache_ratio":0.9171460288626501,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":36352,"estimated_new_tokens":3284,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":978,"cache_hit":0,"new_prefill":39636,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":39636,"score_linear":62618.0,"score_lmetric":39636},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1239,"cache_hit":0,"new_prefill":39636,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":39636,"score_linear":21371.0,"score_lmetric":39636},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":65289,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1134,"cache_hit":0,"new_prefill":39636,"score_linear":65289.0,"score_lmetric":39636},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":41202,"ongoing_decode_tokens":0,"pending_prefill_tokens":3826,"num_requests":1,"active_p_offloads":0,"cached_blocks":1182,"cache_hit":36352,"new_prefill":3284,"score_linear":4850.0,"score_lmetric":7110},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":39636,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":39636,"score_linear":96026.0,"score_lmetric":39636}],"chosen_score_linear":4850.0,"chosen_score_lmetric":7110,"t_first_token":281936.415369556,"t_first_token_unix":1779712801.4293127,"t_done":281936.812054677,"t_done_unix":1779712801.825998},{"request_id":"1348394:1:1348394:481","session_id":"1348394","input_length":17217,"t_proxy_recv":281935.190880218,"t_decision_unix":1779712800.2048218,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":17217,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":978,"cache_hit":0,"new_prefill":17217,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":17217,"score_linear":62618.0,"score_lmetric":17217},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1239,"cache_hit":0,"new_prefill":17217,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":17217,"score_linear":21371.0,"score_lmetric":17217},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":65289,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1134,"cache_hit":0,"new_prefill":17217,"score_linear":65289.0,"score_lmetric":17217},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":80838,"ongoing_decode_tokens":0,"pending_prefill_tokens":7110,"num_requests":2,"active_p_offloads":0,"cached_blocks":1182,"cache_hit":0,"new_prefill":17217,"score_linear":80838.0,"score_lmetric":48654},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":17217,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":17217,"score_linear":96026.0,"score_lmetric":17217}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281936.850808899,"t_first_token_unix":1779712801.8647523,"t_done":281937.227444151,"t_done_unix":1779712802.2413874},{"request_id":"1348691:1:1348691:482","session_id":"1348691","input_length":12701,"t_proxy_recv":281936.047626324,"t_decision_unix":1779712801.0615678,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":12701,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":978,"cache_hit":0,"new_prefill":12701,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":12701,"score_linear":62618.0,"score_lmetric":12701},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1239,"cache_hit":0,"new_prefill":12701,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":12701,"score_linear":21371.0,"score_lmetric":12701},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1262,"cache_hit":0,"new_prefill":12701,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":80838,"ongoing_decode_tokens":41202,"pending_prefill_tokens":3284,"num_requests":2,"active_p_offloads":0,"cached_blocks":1182,"cache_hit":0,"new_prefill":12701,"score_linear":80838.0,"score_lmetric":31970},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":17217,"ongoing_decode_tokens":0,"pending_prefill_tokens":17217,"num_requests":1,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":12701,"score_linear":17217.0,"score_lmetric":29918},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":12701,"score_linear":96026.0,"score_lmetric":12701}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281937.101989562,"t_first_token_unix":1779712802.115933,"t_done":281937.462215144,"t_done_unix":1779712802.4761581},{"request_id":"1344773:3:1349525:486","session_id":"1344773","input_length":33334,"t_proxy_recv":281938.983582555,"t_decision_unix":1779712803.9975238,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":32768,"affinity_cache_ratio":0.9830203395932081,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":32768,"estimated_new_tokens":566,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1002,"cache_hit":32768,"new_prefill":566,"score_linear":-32768.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":33334,"score_linear":62618.0,"score_lmetric":33334},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1239,"cache_hit":0,"new_prefill":33334,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":33334,"score_linear":21371.0,"score_lmetric":33334},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":33334,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":41202,"ongoing_decode_tokens":41202,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1188,"cache_hit":0,"new_prefill":33334,"score_linear":41202.0,"score_lmetric":33334},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":76493,"ongoing_decode_tokens":0,"pending_prefill_tokens":2253,"num_requests":1,"active_p_offloads":0,"cached_blocks":1163,"cache_hit":0,"new_prefill":33334,"score_linear":76493.0,"score_lmetric":35587},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":33334,"score_linear":96026.0,"score_lmetric":33334}],"chosen_score_linear":-32768.0,"chosen_score_lmetric":0,"t_first_token":281939.18568323,"t_first_token_unix":1779712804.1996264,"t_done":281940.627954132,"t_done_unix":1779712805.6418972},{"request_id":"1349975:1:1349975:487","session_id":"1349975","input_length":4988,"t_proxy_recv":281940.57243092,"t_decision_unix":1779712805.5863721,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4988,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":33334,"ongoing_decode_tokens":33334,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1002,"cache_hit":0,"new_prefill":4988,"score_linear":33334.0,"score_lmetric":4988},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":4988,"score_linear":62618.0,"score_lmetric":4988},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1239,"cache_hit":0,"new_prefill":4988,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":4988,"score_linear":21371.0,"score_lmetric":4988},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":4988,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":41202,"ongoing_decode_tokens":41202,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1188,"cache_hit":0,"new_prefill":4988,"score_linear":41202.0,"score_lmetric":4988},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":76493,"ongoing_decode_tokens":76493,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1163,"cache_hit":0,"new_prefill":4988,"score_linear":76493.0,"score_lmetric":4988},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":4988,"score_linear":96026.0,"score_lmetric":4988}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281940.891464314,"t_first_token_unix":1779712805.9054074,"t_done":281941.216759703,"t_done_unix":1779712806.2307034},{"request_id":"1350162:1:1350162:488","session_id":"1350162","input_length":4594,"t_proxy_recv":281941.257237622,"t_decision_unix":1779712806.2711787,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4594,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1003,"cache_hit":0,"new_prefill":4594,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":4594,"score_linear":62618.0,"score_lmetric":4594},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1248,"cache_hit":0,"new_prefill":4594,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":4594,"score_linear":21371.0,"score_lmetric":4594},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":4594,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":41202,"ongoing_decode_tokens":41202,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1188,"cache_hit":0,"new_prefill":4594,"score_linear":41202.0,"score_lmetric":4594},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":76493,"ongoing_decode_tokens":76493,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1163,"cache_hit":0,"new_prefill":4594,"score_linear":76493.0,"score_lmetric":4594},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":4594,"score_linear":96026.0,"score_lmetric":4594}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281941.544394795,"t_first_token_unix":1779712806.5583386,"t_done":281941.661873486,"t_done_unix":1779712806.6758168},{"request_id":"1269373:19:1349406:485","session_id":"1269373","input_length":76493,"t_proxy_recv":281938.637182723,"t_decision_unix":1779712803.6511238,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":74240,"affinity_cache_ratio":0.9705463245002811,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":74240,"estimated_new_tokens":2253,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1002,"cache_hit":7680,"new_prefill":68813,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":76493,"score_linear":62618.0,"score_lmetric":76493},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1239,"cache_hit":0,"new_prefill":76493,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":7680,"new_prefill":68813,"score_linear":13691.0,"score_lmetric":68813},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":76493,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":41202,"ongoing_decode_tokens":41202,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1188,"cache_hit":0,"new_prefill":76493,"score_linear":41202.0,"score_lmetric":76493},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1163,"cache_hit":74240,"new_prefill":2253,"score_linear":-74240.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":7680,"new_prefill":68813,"score_linear":88346.0,"score_lmetric":68813}],"chosen_score_linear":-74240.0,"chosen_score_lmetric":0,"t_first_token":281939.868642419,"t_first_token_unix":1779712804.882586,"t_done":281943.341644695,"t_done_unix":1779712808.3555882},{"request_id":"1350896:1:1350896:489","session_id":"1350896","input_length":1471,"t_proxy_recv":281943.70481674,"t_decision_unix":1779712808.7187579,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1471,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1012,"cache_hit":0,"new_prefill":1471,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":1471,"score_linear":62618.0,"score_lmetric":1471},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1248,"cache_hit":0,"new_prefill":1471,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":1471,"score_linear":21371.0,"score_lmetric":1471},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":1471,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":41202,"ongoing_decode_tokens":41202,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1188,"cache_hit":0,"new_prefill":1471,"score_linear":41202.0,"score_lmetric":1471},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1168,"cache_hit":0,"new_prefill":1471,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":1471,"score_linear":96026.0,"score_lmetric":1471}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281943.806392713,"t_first_token_unix":1779712808.820336,"t_done":281944.172188202,"t_done_unix":1779712809.1861312},{"request_id":"1334223:3:1348241:479","session_id":"1334223","input_length":41202,"t_proxy_recv":281934.588365496,"t_decision_unix":1779712799.6023066,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":37376,"affinity_cache_ratio":0.9071404300762099,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":37376,"estimated_new_tokens":3826,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":978,"cache_hit":0,"new_prefill":41202,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":41202,"score_linear":62618.0,"score_lmetric":41202},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1239,"cache_hit":0,"new_prefill":41202,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":41202,"score_linear":21371.0,"score_lmetric":41202},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":65289,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1134,"cache_hit":0,"new_prefill":41202,"score_linear":65289.0,"score_lmetric":41202},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1182,"cache_hit":37376,"new_prefill":3826,"score_linear":-37376.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":41202,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":41202,"score_linear":96026.0,"score_lmetric":41202}],"chosen_score_linear":-37376.0,"chosen_score_lmetric":0,"t_first_token":281935.595703959,"t_first_token_unix":1779712800.609648,"t_done":281944.635398903,"t_done_unix":1779712809.6493423},{"request_id":"1350980:1:1350980:490","session_id":"1350980","input_length":15313,"t_proxy_recv":281943.960346449,"t_decision_unix":1779712808.9742875,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":15313,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":1471,"ongoing_decode_tokens":1471,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1012,"cache_hit":0,"new_prefill":15313,"score_linear":1471.0,"score_lmetric":15313},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":15313,"score_linear":62618.0,"score_lmetric":15313},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1248,"cache_hit":0,"new_prefill":15313,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":15313,"score_linear":21371.0,"score_lmetric":15313},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":15313,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":41202,"ongoing_decode_tokens":41202,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1188,"cache_hit":0,"new_prefill":15313,"score_linear":41202.0,"score_lmetric":15313},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1168,"cache_hit":0,"new_prefill":15313,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":15313,"score_linear":96026.0,"score_lmetric":15313}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281945.329966495,"t_first_token_unix":1779712810.3439102,"t_done":281945.369265073,"t_done_unix":1779712810.3832085},{"request_id":"1253743:5:1351907:492","session_id":"1253743","input_length":51340,"t_proxy_recv":281947.488350333,"t_decision_unix":1779712812.5022914,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":50176,"affinity_cache_ratio":0.9773276197896377,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":50176,"estimated_new_tokens":1164,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1015,"cache_hit":0,"new_prefill":51340,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":4608,"new_prefill":46732,"score_linear":58010.0,"score_lmetric":46732},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1248,"cache_hit":0,"new_prefill":51340,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":50176,"new_prefill":1164,"score_linear":-28805.0,"score_lmetric":1164},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":51340,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":51340,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":51340,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":51340,"score_linear":96026.0,"score_lmetric":51340}],"chosen_score_linear":-28805.0,"chosen_score_lmetric":1164,"t_first_token":281947.870301135,"t_first_token_unix":1779712812.8842454,"t_done":281948.450087575,"t_done_unix":1779712813.464031},{"request_id":"1350162:2:1352468:495","session_id":"1350162","input_length":10575,"t_proxy_recv":281949.29594628,"t_decision_unix":1779712814.309888,"policy":"unified","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":5,"affinity_cache_hit":4608,"affinity_cache_ratio":0.4357446808510638,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":10575,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55015,"ongoing_decode_tokens":0,"pending_prefill_tokens":9447,"num_requests":2,"active_p_offloads":0,"cached_blocks":1015,"cache_hit":4608,"new_prefill":5967,"score_linear":50407.0,"score_lmetric":30828},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":10575,"score_linear":62618.0,"score_lmetric":10575},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1248,"cache_hit":0,"new_prefill":10575,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":10575,"score_linear":21371.0,"score_lmetric":10575},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":10575,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":10575,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":10575,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":10575,"score_linear":96026.0,"score_lmetric":10575}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281950.121819282,"t_first_token_unix":1779712815.1357625,"t_done":281950.247877638,"t_done_unix":1779712815.261823},{"request_id":"1344773:4:1352462:494","session_id":"1344773","input_length":33579,"t_proxy_recv":281949.284108608,"t_decision_unix":1779712814.2980497,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":33280,"affinity_cache_ratio":0.9910956252419667,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":33280,"estimated_new_tokens":299,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":21436,"ongoing_decode_tokens":0,"pending_prefill_tokens":9148,"num_requests":1,"active_p_offloads":0,"cached_blocks":1015,"cache_hit":33280,"new_prefill":299,"score_linear":-11844.0,"score_lmetric":9447},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":33579,"score_linear":62618.0,"score_lmetric":33579},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1248,"cache_hit":0,"new_prefill":33579,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":33579,"score_linear":21371.0,"score_lmetric":33579},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":33579,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":33579,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":33579,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":33579,"score_linear":96026.0,"score_lmetric":33579}],"chosen_score_linear":-11844.0,"chosen_score_lmetric":9447,"t_first_token":281950.038509401,"t_first_token_unix":1779712815.0524528,"t_done":281950.72890872,"t_done_unix":1779712815.7428522},{"request_id":"1348691:2:1352277:493","session_id":"1348691","input_length":21436,"t_proxy_recv":281948.673166826,"t_decision_unix":1779712813.687108,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":12288,"affinity_cache_ratio":0.5732412763575294,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":12288,"estimated_new_tokens":9148,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1015,"cache_hit":12288,"new_prefill":9148,"score_linear":-12288.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":21436,"score_linear":62618.0,"score_lmetric":21436},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1248,"cache_hit":0,"new_prefill":21436,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":21436,"score_linear":21371.0,"score_lmetric":21436},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":21436,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":21436,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":21436,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":21436,"score_linear":96026.0,"score_lmetric":21436}],"chosen_score_linear":-12288.0,"chosen_score_lmetric":0,"t_first_token":281949.967529099,"t_first_token_unix":1779712814.9814725,"t_done":281950.834332353,"t_done_unix":1779712815.8482754},{"request_id":"1345921:4:1353010:501","session_id":"1345921","input_length":6935,"t_proxy_recv":281951.275816674,"t_decision_unix":1779712816.2897582,"policy":"unified","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":5,"affinity_cache_hit":1024,"affinity_cache_ratio":0.14765681326604183,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":6935,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1032,"cache_hit":0,"new_prefill":6935,"score_linear":68259.0,"score_lmetric":6935},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":6935,"score_linear":62618.0,"score_lmetric":6935},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14239,"ongoing_decode_tokens":0,"pending_prefill_tokens":9631,"num_requests":1,"active_p_offloads":0,"cached_blocks":1248,"cache_hit":0,"new_prefill":6935,"score_linear":14239.0,"score_lmetric":16566},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":6935,"score_linear":21371.0,"score_lmetric":6935},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":102166,"ongoing_decode_tokens":0,"pending_prefill_tokens":19222,"num_requests":2,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":1024,"new_prefill":5911,"score_linear":101142.0,"score_lmetric":50266},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1217,"cache_hit":0,"new_prefill":6935,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":77790,"ongoing_decode_tokens":77790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":6935,"score_linear":77790.0,"score_lmetric":6935},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":512,"new_prefill":6423,"score_linear":95514.0,"score_lmetric":6423}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281951.738490024,"t_first_token_unix":1779712816.7524412,"t_done":281951.740706845,"t_done_unix":1779712816.7546499},{"request_id":"1349975:2:1352986:500","session_id":"1349975","input_length":14239,"t_proxy_recv":281951.236599269,"t_decision_unix":1779712816.2505405,"policy":"unified","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":4608,"affinity_cache_ratio":0.32361823161738884,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":9631,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1032,"cache_hit":0,"new_prefill":14239,"score_linear":68259.0,"score_lmetric":14239},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":14239,"score_linear":62618.0,"score_lmetric":14239},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1248,"cache_hit":4608,"new_prefill":9631,"score_linear":-4608.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":14239,"score_linear":21371.0,"score_lmetric":14239},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":102166,"ongoing_decode_tokens":0,"pending_prefill_tokens":19222,"num_requests":2,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":14239,"score_linear":102166.0,"score_lmetric":66922},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1217,"cache_hit":0,"new_prefill":14239,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":77790,"ongoing_decode_tokens":77790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":14239,"score_linear":77790.0,"score_lmetric":14239},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":14239,"score_linear":96026.0,"score_lmetric":14239}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":281952.212908767,"t_first_token_unix":1779712817.2268524,"t_done":281952.406758139,"t_done_unix":1779712817.4207013},{"request_id":"1352606:1:1352606:496","session_id":"1352606","input_length":18840,"t_proxy_recv":281949.867518481,"t_decision_unix":1779712814.8814592,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":18840,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55015,"ongoing_decode_tokens":0,"pending_prefill_tokens":9447,"num_requests":2,"active_p_offloads":0,"cached_blocks":1015,"cache_hit":0,"new_prefill":18840,"score_linear":55015.0,"score_lmetric":56574},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":18840,"score_linear":62618.0,"score_lmetric":18840},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1248,"cache_hit":0,"new_prefill":18840,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":18840,"score_linear":21371.0,"score_lmetric":18840},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":18840,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":10575,"ongoing_decode_tokens":0,"pending_prefill_tokens":10575,"num_requests":1,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":18840,"score_linear":10575.0,"score_lmetric":29415},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":18840,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":18840,"score_linear":96026.0,"score_lmetric":18840}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281951.75954958,"t_first_token_unix":1779712816.7734928,"t_done":281952.768847028,"t_done_unix":1779712817.7827904},{"request_id":"1317598:6:1352957:499","session_id":"1317598","input_length":83326,"t_proxy_recv":281951.136154564,"t_decision_unix":1779712816.1500962,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":82944,"affinity_cache_ratio":0.9954155965724983,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":82944,"estimated_new_tokens":382,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1032,"cache_hit":0,"new_prefill":83326,"score_linear":68259.0,"score_lmetric":83326},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":83326,"score_linear":62618.0,"score_lmetric":83326},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1248,"cache_hit":0,"new_prefill":83326,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":83326,"score_linear":21371.0,"score_lmetric":83326},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18840,"ongoing_decode_tokens":0,"pending_prefill_tokens":18840,"num_requests":1,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":82944,"new_prefill":382,"score_linear":-64104.0,"score_lmetric":19222},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1217,"cache_hit":0,"new_prefill":83326,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":77790,"ongoing_decode_tokens":77790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":83326,"score_linear":77790.0,"score_lmetric":83326},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":83326,"score_linear":96026.0,"score_lmetric":83326}],"chosen_score_linear":-64104.0,"chosen_score_lmetric":19222,"t_first_token":281951.990753554,"t_first_token_unix":1779712817.0046968,"t_done":281953.1474425,"t_done_unix":1779712818.161386},{"request_id":"1269373:20:1352797:498","session_id":"1269373","input_length":77790,"t_proxy_recv":281950.538216507,"t_decision_unix":1779712815.5521579,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":76800,"affinity_cache_ratio":0.9872734284612418,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":76800,"estimated_new_tokens":990,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":123274,"ongoing_decode_tokens":123274,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1015,"cache_hit":7680,"new_prefill":70110,"score_linear":115594.0,"score_lmetric":210330},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":77790,"score_linear":62618.0,"score_lmetric":77790},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1248,"cache_hit":0,"new_prefill":77790,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":7680,"new_prefill":70110,"score_linear":13691.0,"score_lmetric":70110},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18840,"ongoing_decode_tokens":0,"pending_prefill_tokens":18840,"num_requests":1,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":77790,"score_linear":18840.0,"score_lmetric":96630},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1217,"cache_hit":0,"new_prefill":77790,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":76800,"new_prefill":990,"score_linear":-76800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":7680,"new_prefill":70110,"score_linear":88346.0,"score_lmetric":70110}],"chosen_score_linear":-76800.0,"chosen_score_lmetric":0,"t_first_token":281951.101368328,"t_first_token_unix":1779712816.1153119,"t_done":281953.201167733,"t_done_unix":1779712818.2151105},{"request_id":"1279412:23:1348089:477","session_id":"1279412","input_length":96026,"t_proxy_recv":281933.972769292,"t_decision_unix":1779712798.9867105,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":95744,"affinity_cache_ratio":0.9970632953575074,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":95744,"estimated_new_tokens":282,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":978,"cache_hit":0,"new_prefill":96026,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":96026,"score_linear":62618.0,"score_lmetric":96026},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1239,"cache_hit":0,"new_prefill":96026,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":96026,"score_linear":21371.0,"score_lmetric":96026},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65289,"ongoing_decode_tokens":65289,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1134,"cache_hit":0,"new_prefill":96026,"score_linear":65289.0,"score_lmetric":96026},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1182,"cache_hit":0,"new_prefill":96026,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":96026,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":95744,"new_prefill":282,"score_linear":-95744.0,"score_lmetric":0}],"chosen_score_linear":-95744.0,"chosen_score_lmetric":0,"t_first_token":281934.144053018,"t_first_token_unix":1779712799.1579967,"t_done":281954.33326602,"t_done_unix":1779712819.3472092},{"request_id":"1344773:5:1353365:502","session_id":"1344773","input_length":33875,"t_proxy_recv":281952.741386263,"t_decision_unix":1779712817.7553275,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":33280,"affinity_cache_ratio":0.9824354243542436,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":33280,"estimated_new_tokens":595,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1032,"cache_hit":33280,"new_prefill":595,"score_linear":34979.0,"score_lmetric":595},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":33875,"score_linear":62618.0,"score_lmetric":33875},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":33875,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":33875,"score_linear":21371.0,"score_lmetric":33875},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":102166,"ongoing_decode_tokens":102166,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":33875,"score_linear":102166.0,"score_lmetric":67750},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":33875,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":77790,"ongoing_decode_tokens":77790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":33875,"score_linear":77790.0,"score_lmetric":33875},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":33875,"score_linear":96026.0,"score_lmetric":33875}],"chosen_score_linear":34979.0,"chosen_score_lmetric":595,"t_first_token":281952.936787186,"t_first_token_unix":1779712817.9507306,"t_done":281955.957773363,"t_done_unix":1779712820.9717166},{"request_id":"1253743:6:1354346:505","session_id":"1253743","input_length":51436,"t_proxy_recv":281955.988753722,"t_decision_unix":1779712821.002695,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":51200,"affinity_cache_ratio":0.9954117738548877,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":51200,"estimated_new_tokens":236,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1033,"cache_hit":0,"new_prefill":51436,"score_linear":68259.0,"score_lmetric":51436},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":4608,"new_prefill":46828,"score_linear":58010.0,"score_lmetric":46828},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":27651,"ongoing_decode_tokens":0,"pending_prefill_tokens":27651,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":51436,"score_linear":27651.0,"score_lmetric":79087},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":51200,"new_prefill":236,"score_linear":-29829.0,"score_lmetric":236},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":51436,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":51436,"score_linear":83526.0,"score_lmetric":134962},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":0,"new_prefill":51436,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1204,"cache_hit":0,"new_prefill":51436,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-29829.0,"chosen_score_lmetric":236,"t_first_token":281956.088321462,"t_first_token_unix":1779712821.1022646,"t_done":281956.649657819,"t_done_unix":1779712821.6636012},{"request_id":"1353821:1:1353821:503","session_id":"1353821","input_length":27651,"t_proxy_recv":281954.158612886,"t_decision_unix":1779712819.1725543,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":27651,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":102134,"ongoing_decode_tokens":102134,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1032,"cache_hit":0,"new_prefill":27651,"score_linear":102134.0,"score_lmetric":55302},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":27651,"score_linear":62618.0,"score_lmetric":27651},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":27651,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":27651,"score_linear":21371.0,"score_lmetric":27651},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":27651,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":27651,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":0,"new_prefill":27651,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":27651,"score_linear":96026.0,"score_lmetric":27651}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281957.589890656,"t_first_token_unix":1779712822.6038342,"t_done":281958.402039571,"t_done_unix":1779712823.4159825},{"request_id":"1355484:1:1355484:508","session_id":"1355484","input_length":19366,"t_proxy_recv":281959.865541302,"t_decision_unix":1779712824.8794823,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":6566,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":83778,"ongoing_decode_tokens":83778,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1033,"cache_hit":0,"new_prefill":19366,"score_linear":83778.0,"score_lmetric":38732},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":19366,"score_linear":62618.0,"score_lmetric":19366},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":19366,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":19366,"score_linear":21371.0,"score_lmetric":19366},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":19366,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":19366,"score_linear":83526.0,"score_lmetric":102892},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24503,"ongoing_decode_tokens":0,"pending_prefill_tokens":24503,"num_requests":1,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":0,"new_prefill":19366,"score_linear":24503.0,"score_lmetric":43869},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1204,"cache_hit":12800,"new_prefill":6566,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-12800.0,"chosen_score_lmetric":0,"t_first_token":281960.794792715,"t_first_token_unix":1779712825.8087358,"t_done":281961.395243562,"t_done_unix":1779712826.4091868},{"request_id":"1286804:3:1354910:506","session_id":"1286804","input_length":15519,"t_proxy_recv":281957.960405547,"t_decision_unix":1779712822.974347,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":13824,"affinity_cache_ratio":0.8907790450415619,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":13824,"estimated_new_tokens":1695,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1033,"cache_hit":13824,"new_prefill":1695,"score_linear":54435.0,"score_lmetric":1695},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":15519,"score_linear":62618.0,"score_lmetric":15519},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":27651,"ongoing_decode_tokens":27651,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":15519,"score_linear":27651.0,"score_lmetric":15519},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":15519,"score_linear":21371.0,"score_lmetric":15519},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":15519,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":15519,"score_linear":83526.0,"score_lmetric":99045},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":0,"new_prefill":15519,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1204,"cache_hit":0,"new_prefill":15519,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":54435.0,"chosen_score_lmetric":1695,"t_first_token":281959.302852619,"t_first_token_unix":1779712824.3167984,"t_done":281962.147677176,"t_done_unix":1779712827.1616201},{"request_id":"1355325:1:1355325:507","session_id":"1355325","input_length":24503,"t_proxy_recv":281959.434436272,"t_decision_unix":1779712824.4483776,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":24503,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":83778,"ongoing_decode_tokens":83778,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1033,"cache_hit":0,"new_prefill":24503,"score_linear":83778.0,"score_lmetric":49006},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":24503,"score_linear":62618.0,"score_lmetric":24503},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":24503,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":24503,"score_linear":21371.0,"score_lmetric":24503},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":24503,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":24503,"score_linear":83526.0,"score_lmetric":108029},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":0,"new_prefill":24503,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1204,"cache_hit":0,"new_prefill":24503,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281962.257796803,"t_first_token_unix":1779712827.271743,"t_done":281962.4272478,"t_done_unix":1779712827.441191},{"request_id":"1344773:6:1356097:513","session_id":"1344773","input_length":34198,"t_proxy_recv":281962.142837335,"t_decision_unix":1779712827.156777,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9881279606994561,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":33792,"estimated_new_tokens":406,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":83778,"ongoing_decode_tokens":83778,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1033,"cache_hit":33792,"new_prefill":406,"score_linear":49986.0,"score_lmetric":812},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":110811,"ongoing_decode_tokens":110811,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":34198,"score_linear":110811.0,"score_lmetric":68396},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":34198,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":34198,"score_linear":21371.0,"score_lmetric":34198},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":34198,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":34198,"score_linear":83526.0,"score_lmetric":117724},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":102592,"ongoing_decode_tokens":0,"pending_prefill_tokens":24768,"num_requests":2,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":0,"new_prefill":34198,"score_linear":102592.0,"score_lmetric":117932},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":84285,"ongoing_decode_tokens":0,"pending_prefill_tokens":36157,"num_requests":2,"active_p_offloads":0,"cached_blocks":1217,"cache_hit":0,"new_prefill":34198,"score_linear":84285.0,"score_lmetric":140710}],"chosen_score_linear":49986.0,"chosen_score_lmetric":812,"t_first_token":281962.241593273,"t_first_token_unix":1779712827.2555368,"t_done":281962.501779225,"t_done_unix":1779712827.5157223},{"request_id":"1253804:6:1318560:340","session_id":"1253804","input_length":62618,"t_proxy_recv":281829.785578702,"t_decision_unix":1779712694.7995195,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9403047047174934,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":3738,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":779,"cache_hit":0,"new_prefill":62618,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":943,"cache_hit":58880,"new_prefill":3738,"score_linear":-58880.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":62618,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":3584,"new_prefill":59034,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":62618,"score_linear":79663.0,"score_lmetric":141769},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":837,"cache_hit":0,"new_prefill":62618,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":772,"cache_hit":0,"new_prefill":62618,"score_linear":188103.0,"score_lmetric":125236},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":62618,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58880.0,"chosen_score_lmetric":0,"t_first_token":281833.688998404,"t_first_token_unix":1779712698.702955,"t_done":281963.073674385,"t_done_unix":1779712828.0876224},{"request_id":"1356948:1:1356948:515","session_id":"1356948","input_length":6711,"t_proxy_recv":281965.150660439,"t_decision_unix":1779712830.1646016,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":6711,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1036,"cache_hit":0,"new_prefill":6711,"score_linear":68259.0,"score_lmetric":6711},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":48193,"ongoing_decode_tokens":48193,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":982,"cache_hit":0,"new_prefill":6711,"score_linear":48193.0,"score_lmetric":6711},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":122597,"ongoing_decode_tokens":0,"pending_prefill_tokens":122597,"num_requests":1,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":6711,"score_linear":122597.0,"score_lmetric":129308},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":6711,"score_linear":21371.0,"score_lmetric":6711},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":6711,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":6711,"score_linear":83526.0,"score_lmetric":90237},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78089,"ongoing_decode_tokens":78089,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":6711,"score_linear":78089.0,"score_lmetric":6711},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":84285,"ongoing_decode_tokens":0,"pending_prefill_tokens":36157,"num_requests":2,"active_p_offloads":0,"cached_blocks":1217,"cache_hit":0,"new_prefill":6711,"score_linear":84285.0,"score_lmetric":85736}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281965.603172467,"t_first_token_unix":1779712830.6171167,"t_done":281965.879412079,"t_done_unix":1779712830.8933554},{"request_id":"1269373:21:1355702:509","session_id":"1269373","input_length":78089,"t_proxy_recv":281960.556659502,"t_decision_unix":1779712825.5706007,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":77824,"affinity_cache_ratio":0.9966064362458221,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":77824,"estimated_new_tokens":265,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":83778,"ongoing_decode_tokens":83778,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1033,"cache_hit":7680,"new_prefill":70409,"score_linear":76098.0,"score_lmetric":140818},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":78089,"score_linear":62618.0,"score_lmetric":78089},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":78089,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":7680,"new_prefill":70409,"score_linear":13691.0,"score_lmetric":70409},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":78089,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":78089,"score_linear":83526.0,"score_lmetric":161615},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24503,"ongoing_decode_tokens":0,"pending_prefill_tokens":24503,"num_requests":1,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":77824,"new_prefill":265,"score_linear":-53321.0,"score_lmetric":24768},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":19366,"ongoing_decode_tokens":0,"pending_prefill_tokens":6566,"num_requests":1,"active_p_offloads":0,"cached_blocks":1204,"cache_hit":7680,"new_prefill":70409,"score_linear":11686.0,"score_lmetric":76975}],"chosen_score_linear":-53321.0,"chosen_score_lmetric":24768,"t_first_token":281962.376630239,"t_first_token_unix":1779712827.3905735,"t_done":281966.131065128,"t_done_unix":1779712831.1450086},{"request_id":"1355951:1:1355951:511","session_id":"1355951","input_length":35308,"t_proxy_recv":281961.46008632,"t_decision_unix":1779712826.4740279,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":35308,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":83778,"ongoing_decode_tokens":83778,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1033,"cache_hit":0,"new_prefill":35308,"score_linear":83778.0,"score_lmetric":70616},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":110811,"ongoing_decode_tokens":62618,"pending_prefill_tokens":3137,"num_requests":2,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":35308,"score_linear":110811.0,"score_lmetric":76890},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":35308,"score_linear":21371.0,"score_lmetric":35308},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":35308,"score_linear":83526.0,"score_lmetric":118834},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":102592,"ongoing_decode_tokens":0,"pending_prefill_tokens":24768,"num_requests":2,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":0,"new_prefill":35308,"score_linear":102592.0,"score_lmetric":120152},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1217,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281966.785454266,"t_first_token_unix":1779712831.799398,"t_done":281967.332367596,"t_done_unix":1779712832.3463104},{"request_id":"1340278:2:1356079:512","session_id":"1340278","input_length":48977,"t_proxy_recv":281962.059832222,"t_decision_unix":1779712827.0737736,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":48128,"affinity_cache_ratio":0.9826653327071891,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":48128,"estimated_new_tokens":849,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":83778,"ongoing_decode_tokens":83778,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1033,"cache_hit":0,"new_prefill":48977,"score_linear":83778.0,"score_lmetric":97954},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":110811,"ongoing_decode_tokens":110811,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":48977,"score_linear":110811.0,"score_lmetric":97954},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":48977,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":48977,"score_linear":21371.0,"score_lmetric":48977},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":48977,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":48977,"score_linear":83526.0,"score_lmetric":132503},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":102592,"ongoing_decode_tokens":0,"pending_prefill_tokens":24768,"num_requests":2,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":0,"new_prefill":48977,"score_linear":102592.0,"score_lmetric":147490},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":35308,"ongoing_decode_tokens":0,"pending_prefill_tokens":35308,"num_requests":1,"active_p_offloads":0,"cached_blocks":1217,"cache_hit":48128,"new_prefill":849,"score_linear":-12820.0,"score_lmetric":36157}],"chosen_score_linear":-12820.0,"chosen_score_lmetric":36157,"t_first_token":281966.788436594,"t_first_token_unix":1779712831.8023798,"t_done":281967.571767295,"t_done_unix":1779712832.5857108},{"request_id":"1357132:1:1357132:516","session_id":"1357132","input_length":13520,"t_proxy_recv":281965.69741579,"t_decision_unix":1779712830.711357,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":13520,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":13520,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1036,"cache_hit":0,"new_prefill":13520,"score_linear":68259.0,"score_lmetric":13520},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":48193,"ongoing_decode_tokens":48193,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":982,"cache_hit":0,"new_prefill":13520,"score_linear":48193.0,"score_lmetric":13520},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":122597,"ongoing_decode_tokens":0,"pending_prefill_tokens":122597,"num_requests":1,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":13520,"score_linear":122597.0,"score_lmetric":136117},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":13520,"score_linear":21371.0,"score_lmetric":13520},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":6711,"ongoing_decode_tokens":6711,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":13520,"score_linear":6711.0,"score_lmetric":13520},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":13520,"score_linear":83526.0,"score_lmetric":97046},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78089,"ongoing_decode_tokens":78089,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":13520,"score_linear":78089.0,"score_lmetric":13520},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":84285,"ongoing_decode_tokens":0,"pending_prefill_tokens":36157,"num_requests":2,"active_p_offloads":0,"cached_blocks":1217,"cache_hit":0,"new_prefill":13520,"score_linear":84285.0,"score_lmetric":99354}],"chosen_score_linear":21371.0,"chosen_score_lmetric":13520,"t_first_token":281966.859038195,"t_first_token_unix":1779712831.8729818,"t_done":281967.750009743,"t_done_unix":1779712832.763953},{"request_id":"1344773:7:1357714:518","session_id":"1344773","input_length":34351,"t_proxy_recv":281967.624628916,"t_decision_unix":1779712832.6385698,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9837268201799074,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":33792,"estimated_new_tokens":559,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1036,"cache_hit":33792,"new_prefill":559,"score_linear":34467.0,"score_lmetric":559},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":48193,"ongoing_decode_tokens":48193,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":982,"cache_hit":0,"new_prefill":34351,"score_linear":48193.0,"score_lmetric":34351},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":34351,"score_linear":150402.0,"score_lmetric":314210},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":34891,"ongoing_decode_tokens":34891,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":34351,"score_linear":34891.0,"score_lmetric":68702},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":34351,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":34351,"score_linear":83526.0,"score_lmetric":117877},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":34351,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":0,"new_prefill":34351,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":34467.0,"chosen_score_lmetric":559,"t_first_token":281967.752417171,"t_first_token_unix":1779712832.76636,"t_done":281968.41783933,"t_done_unix":1779712833.4317825},{"request_id":"1294611:11:1355750:510","session_id":"1294611","input_length":48193,"t_proxy_recv":281960.645183619,"t_decision_unix":1779712825.6591249,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":45056,"affinity_cache_ratio":0.9349075591890939,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":45056,"estimated_new_tokens":3137,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":83778,"ongoing_decode_tokens":83778,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1033,"cache_hit":0,"new_prefill":48193,"score_linear":83778.0,"score_lmetric":96386},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":45056,"new_prefill":3137,"score_linear":17562.0,"score_lmetric":3137},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":12800,"new_prefill":35393,"score_linear":-12800.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":12800,"new_prefill":35393,"score_linear":8571.0,"score_lmetric":35393},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":48193,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":48193,"score_linear":83526.0,"score_lmetric":131719},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":102592,"ongoing_decode_tokens":0,"pending_prefill_tokens":24768,"num_requests":2,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":0,"new_prefill":48193,"score_linear":102592.0,"score_lmetric":145922},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":19366,"ongoing_decode_tokens":0,"pending_prefill_tokens":6566,"num_requests":1,"active_p_offloads":0,"cached_blocks":1204,"cache_hit":0,"new_prefill":48193,"score_linear":19366.0,"score_lmetric":54759}],"chosen_score_linear":17562.0,"chosen_score_lmetric":3137,"t_first_token":281961.567636168,"t_first_token_unix":1779712826.5815794,"t_done":281969.651417444,"t_done_unix":1779712834.6653626},{"request_id":"1344773:8:1358989:523","session_id":"1344773","input_length":34552,"t_proxy_recv":281972.150506038,"t_decision_unix":1779712837.1644473,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9928224125955082,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34304,"estimated_new_tokens":248,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":34304,"new_prefill":248,"score_linear":33955.0,"score_lmetric":248},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":989,"cache_hit":0,"new_prefill":34552,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":34552,"score_linear":150402.0,"score_lmetric":314612},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1321,"cache_hit":0,"new_prefill":34552,"score_linear":21371.0,"score_lmetric":34552},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":34552,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":34552,"score_linear":83526.0,"score_lmetric":118078},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":0,"pending_prefill_tokens":1910,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":34552,"score_linear":80246.0,"score_lmetric":36462},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":34694,"num_requests":1,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":0,"new_prefill":34552,"score_linear":58758.0,"score_lmetric":69246}],"chosen_score_linear":33955.0,"chosen_score_lmetric":248,"t_first_token":281972.278658728,"t_first_token_unix":1779712837.2926018,"t_done":281972.710237236,"t_done_unix":1779712837.7241805},{"request_id":"1253743:7:1359251:524","session_id":"1253743","input_length":51509,"t_proxy_recv":281973.186909184,"t_decision_unix":1779712838.2008505,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":51200,"affinity_cache_ratio":0.9940010483604806,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":51200,"estimated_new_tokens":309,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":0,"new_prefill":51509,"score_linear":68259.0,"score_lmetric":51509},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":989,"cache_hit":4608,"new_prefill":46901,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":51509,"score_linear":150402.0,"score_lmetric":348526},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1321,"cache_hit":51200,"new_prefill":309,"score_linear":-29829.0,"score_lmetric":309},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":51509,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":51509,"score_linear":83526.0,"score_lmetric":135035},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":51509,"score_linear":80246.0,"score_lmetric":51509},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":34694,"num_requests":1,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":0,"new_prefill":51509,"score_linear":58758.0,"score_lmetric":86203}],"chosen_score_linear":-29829.0,"chosen_score_lmetric":309,"t_first_token":281973.278717506,"t_first_token_unix":1779712838.2926607,"t_done":281973.511415834,"t_done_unix":1779712838.525359},{"request_id":"1344773:9:1360064:526","session_id":"1344773","input_length":34611,"t_proxy_recv":281976.156268433,"t_decision_unix":1779712841.17021,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9911299875762041,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34304,"estimated_new_tokens":307,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":34304,"new_prefill":307,"score_linear":33955.0,"score_lmetric":307},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":989,"cache_hit":0,"new_prefill":34611,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":34611,"score_linear":150402.0,"score_lmetric":314730},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1321,"cache_hit":0,"new_prefill":34611,"score_linear":21371.0,"score_lmetric":34611},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":34611,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":34611,"score_linear":83526.0,"score_lmetric":118137},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":34611,"score_linear":80246.0,"score_lmetric":34611},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":78251,"ongoing_decode_tokens":0,"pending_prefill_tokens":34731,"num_requests":2,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":0,"new_prefill":34611,"score_linear":78251.0,"score_lmetric":138684}],"chosen_score_linear":33955.0,"chosen_score_lmetric":307,"t_first_token":281976.228261902,"t_first_token_unix":1779712841.242205,"t_done":281976.902396622,"t_done_unix":1779712841.9163396},{"request_id":"1336620:3:1339999:443","session_id":"1336620","input_length":21371,"t_proxy_recv":281909.816521919,"t_decision_unix":1779712774.8304634,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":14848,"affinity_cache_ratio":0.6947732909082401,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":14848,"estimated_new_tokens":6523,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":121282,"ongoing_decode_tokens":121282,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":900,"cache_hit":7680,"new_prefill":13691,"score_linear":113602.0,"score_lmetric":27382},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":21371,"score_linear":62618.0,"score_lmetric":21371},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1215,"cache_hit":0,"new_prefill":21371,"score_linear":7065.0,"score_lmetric":21371},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":99903,"ongoing_decode_tokens":99903,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":14848,"new_prefill":6523,"score_linear":85055.0,"score_lmetric":13046},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":2279,"ongoing_decode_tokens":2279,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1080,"cache_hit":0,"new_prefill":21371,"score_linear":2279.0,"score_lmetric":21371},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":5280,"ongoing_decode_tokens":5280,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1081,"cache_hit":0,"new_prefill":21371,"score_linear":5280.0,"score_lmetric":21371},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":45683,"ongoing_decode_tokens":45683,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":7680,"new_prefill":13691,"score_linear":38003.0,"score_lmetric":27382},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1082,"cache_hit":7680,"new_prefill":13691,"score_linear":40701.0,"score_lmetric":62072}],"chosen_score_linear":85055.0,"chosen_score_lmetric":13046,"t_first_token":281910.803675282,"t_first_token_unix":1779712775.8176215,"t_done":281977.132952876,"t_done_unix":1779712842.1468961},{"request_id":"1353951:1:1353951:504","session_id":"1353951","input_length":83526,"t_proxy_recv":281954.658162762,"t_decision_unix":1779712819.6721041,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":83526,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":102134,"ongoing_decode_tokens":102134,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1032,"cache_hit":0,"new_prefill":83526,"score_linear":102134.0,"score_lmetric":167052},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":83526,"score_linear":62618.0,"score_lmetric":83526},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":27651,"ongoing_decode_tokens":0,"pending_prefill_tokens":27651,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":83526,"score_linear":27651.0,"score_lmetric":111177},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":83526,"score_linear":21371.0,"score_lmetric":83526},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":83526,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":83526,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1199,"cache_hit":0,"new_prefill":83526,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1204,"cache_hit":0,"new_prefill":83526,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281978.198110554,"t_first_token_unix":1779712843.2120554,"t_done":281979.58778083,"t_done_unix":1779712844.6017241},{"request_id":"1353951:2:1358300:520","session_id":"1353951","input_length":83905,"t_proxy_recv":281979.602950226,"t_decision_unix":1779712844.6168916,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":83456,"affinity_cache_ratio":0.9946487098504261,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":83456,"estimated_new_tokens":449,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":0,"new_prefill":83905,"score_linear":68259.0,"score_lmetric":83905},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":989,"cache_hit":0,"new_prefill":83905,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":83905,"score_linear":150402.0,"score_lmetric":413318},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":83905,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":183920,"ongoing_decode_tokens":0,"pending_prefill_tokens":118384,"num_requests":2,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":83905,"score_linear":183920.0,"score_lmetric":404578},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":83456,"new_prefill":449,"score_linear":-83456.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":83905,"score_linear":80246.0,"score_lmetric":83905},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":111729,"ongoing_decode_tokens":0,"pending_prefill_tokens":40049,"num_requests":3,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":0,"new_prefill":83905,"score_linear":111729.0,"score_lmetric":371862}],"chosen_score_linear":-83456.0,"chosen_score_lmetric":0,"t_first_token":281979.877970768,"t_first_token_unix":1779712844.8919146,"t_done":281980.026953923,"t_done_unix":1779712845.0408974},{"request_id":"1356948:2:1361262:532","session_id":"1356948","input_length":16480,"t_proxy_recv":281980.436511365,"t_decision_unix":1779712845.4504526,"policy":"unified","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":1,"affinity_cache_hit":6656,"affinity_cache_ratio":0.40388349514563104,"affinity_num_requests":2,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":16480,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":0,"new_prefill":16480,"score_linear":68259.0,"score_lmetric":16480},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":989,"cache_hit":0,"new_prefill":16480,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":16480,"score_linear":150402.0,"score_lmetric":278468},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":16480,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":183920,"ongoing_decode_tokens":0,"pending_prefill_tokens":118384,"num_requests":2,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":6656,"new_prefill":9824,"score_linear":177264.0,"score_lmetric":256416},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":16480,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":16480,"score_linear":80246.0,"score_lmetric":16480},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":111729,"ongoing_decode_tokens":78251,"pending_prefill_tokens":5318,"num_requests":3,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":0,"new_prefill":16480,"score_linear":111729.0,"score_lmetric":65394}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281981.977846515,"t_first_token_unix":1779712846.99179,"t_done":281982.143351972,"t_done_unix":1779712847.157295},{"request_id":"1361992:1:1361992:535","session_id":"1361992","input_length":8113,"t_proxy_recv":281983.013376207,"t_decision_unix":1779712848.027317,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8113,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":102938,"ongoing_decode_tokens":102938,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":0,"new_prefill":8113,"score_linear":102938.0,"score_lmetric":16226},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1021,"cache_hit":0,"new_prefill":8113,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":8113,"score_linear":150402.0,"score_lmetric":261734},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":8113,"score_linear":76969.0,"score_lmetric":85082},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":183920,"ongoing_decode_tokens":0,"pending_prefill_tokens":118384,"num_requests":2,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":8113,"score_linear":183920.0,"score_lmetric":252994},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":8113,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":8113,"score_linear":80246.0,"score_lmetric":8113},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":111729,"ongoing_decode_tokens":78251,"pending_prefill_tokens":5318,"num_requests":3,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":0,"new_prefill":8113,"score_linear":111729.0,"score_lmetric":40293}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281983.580664811,"t_first_token_unix":1779712848.594608,"t_done":281983.719712657,"t_done_unix":1779712848.7336557},{"request_id":"1344773:10:1361326:533","session_id":"1344773","input_length":34679,"t_proxy_recv":281980.680531856,"t_decision_unix":1779712845.6944733,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9891865394042504,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":34304,"estimated_new_tokens":375,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":34304,"new_prefill":375,"score_linear":33955.0,"score_lmetric":375},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":16480,"ongoing_decode_tokens":0,"pending_prefill_tokens":16480,"num_requests":1,"active_p_offloads":0,"cached_blocks":989,"cache_hit":0,"new_prefill":34679,"score_linear":16480.0,"score_lmetric":51159},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":34679,"score_linear":150402.0,"score_lmetric":314866},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":34679,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":183920,"ongoing_decode_tokens":0,"pending_prefill_tokens":118384,"num_requests":2,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":34679,"score_linear":183920.0,"score_lmetric":306126},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":34679,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":34679,"score_linear":80246.0,"score_lmetric":34679},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":111729,"ongoing_decode_tokens":78251,"pending_prefill_tokens":5318,"num_requests":3,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":0,"new_prefill":34679,"score_linear":111729.0,"score_lmetric":119991}],"chosen_score_linear":33955.0,"chosen_score_lmetric":375,"t_first_token":281980.756829728,"t_first_token_unix":1779712845.7707732,"t_done":281984.794225552,"t_done_unix":1779712849.8081706},{"request_id":"1362265:1:1362265:536","session_id":"1362265","input_length":8825,"t_proxy_recv":281983.859995692,"t_decision_unix":1779712848.8739371,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8825,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":102938,"ongoing_decode_tokens":102938,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":0,"new_prefill":8825,"score_linear":102938.0,"score_lmetric":17650},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1021,"cache_hit":0,"new_prefill":8825,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":8825,"score_linear":150402.0,"score_lmetric":263158},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":8825,"score_linear":76969.0,"score_lmetric":85794},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":183920,"ongoing_decode_tokens":0,"pending_prefill_tokens":118384,"num_requests":2,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":8825,"score_linear":183920.0,"score_lmetric":254418},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1408,"cache_hit":0,"new_prefill":8825,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":8825,"score_linear":80246.0,"score_lmetric":8825},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":111729,"ongoing_decode_tokens":78251,"pending_prefill_tokens":5318,"num_requests":3,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":0,"new_prefill":8825,"score_linear":111729.0,"score_lmetric":42429}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281984.529188108,"t_first_token_unix":1779712849.5431316,"t_done":281985.599330861,"t_done_unix":1779712850.613274},{"request_id":"1270340:3:1360716:528","session_id":"1270340","input_length":33478,"t_proxy_recv":281978.300346418,"t_decision_unix":1779712843.3142877,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":28160,"affinity_cache_ratio":0.8411494115538563,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":28160,"estimated_new_tokens":5318,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":0,"new_prefill":33478,"score_linear":68259.0,"score_lmetric":33478},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":989,"cache_hit":0,"new_prefill":33478,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":33478,"score_linear":150402.0,"score_lmetric":312464},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":33478,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":1536,"new_prefill":31942,"score_linear":116352.0,"score_lmetric":149830},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":83526,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":33478,"score_linear":83526.0,"score_lmetric":33478},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":33478,"score_linear":80246.0,"score_lmetric":33478},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":78251,"ongoing_decode_tokens":0,"pending_prefill_tokens":34731,"num_requests":2,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":28160,"new_prefill":5318,"score_linear":50091.0,"score_lmetric":80098}],"chosen_score_linear":50091.0,"chosen_score_lmetric":80098,"t_first_token":281984.936165225,"t_first_token_unix":1779712849.9501085,"t_done":281986.204803749,"t_done_unix":1779712851.218747},{"request_id":"1361992:2:1362904:537","session_id":"1361992","input_length":10347,"t_proxy_recv":281986.104247722,"t_decision_unix":1779712851.118189,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":7680,"affinity_cache_ratio":0.7422441287329661,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":2667,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":0,"new_prefill":10347,"score_linear":68259.0,"score_lmetric":10347},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":0,"new_prefill":10347,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":10347,"score_linear":150402.0,"score_lmetric":266202},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":10347,"score_linear":76969.0,"score_lmetric":87316},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":183920,"ongoing_decode_tokens":0,"pending_prefill_tokens":118384,"num_requests":2,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":10347,"score_linear":183920.0,"score_lmetric":257462},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1408,"cache_hit":7680,"new_prefill":2667,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":10347,"score_linear":80246.0,"score_lmetric":10347},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":111729,"ongoing_decode_tokens":111729,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":0,"new_prefill":10347,"score_linear":111729.0,"score_lmetric":31041}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":281986.363971354,"t_first_token_unix":1779712851.377915,"t_done":281986.509854146,"t_done_unix":1779712851.5237975},{"request_id":"1355484:2:1359370:525","session_id":"1355484","input_length":19493,"t_proxy_recv":281973.641884962,"t_decision_unix":1779712838.6558263,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":19456,"affinity_cache_ratio":0.9981018827271328,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":19456,"estimated_new_tokens":37,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":0,"new_prefill":19493,"score_linear":68259.0,"score_lmetric":19493},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":989,"cache_hit":0,"new_prefill":19493,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":19493,"score_linear":150402.0,"score_lmetric":284494},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1321,"cache_hit":0,"new_prefill":19493,"score_linear":21371.0,"score_lmetric":19493},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":19493,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":19493,"score_linear":83526.0,"score_lmetric":103019},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":19493,"score_linear":80246.0,"score_lmetric":19493},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":34694,"num_requests":1,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":19456,"new_prefill":37,"score_linear":39302.0,"score_lmetric":34731}],"chosen_score_linear":39302.0,"chosen_score_lmetric":34731,"t_first_token":281980.25889846,"t_first_token_unix":1779712845.2728417,"t_done":281987.236723277,"t_done_unix":1779712852.2506669},{"request_id":"1363440:1:1363440:542","session_id":"1363440","input_length":543,"t_proxy_recv":281988.05378015,"t_decision_unix":1779712853.0677218,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":543,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":0,"new_prefill":543,"score_linear":68259.0,"score_lmetric":543},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":0,"new_prefill":543,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":543,"score_linear":150402.0,"score_lmetric":246594},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":543,"score_linear":76969.0,"score_lmetric":77512},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":543,"score_linear":199977.0,"score_lmetric":363480},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":543,"score_linear":62685.0,"score_lmetric":63228},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":543,"score_linear":80246.0,"score_lmetric":543},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":115704,"ongoing_decode_tokens":78472,"pending_prefill_tokens":1904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1297,"cache_hit":0,"new_prefill":543,"score_linear":115704.0,"score_lmetric":7341}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281988.112719823,"t_first_token_unix":1779712853.1266632,"t_done":281988.291743324,"t_done_unix":1779712853.3056872},{"request_id":"1355484:3:1361065:531","session_id":"1355484","input_length":19714,"t_proxy_recv":281987.241619477,"t_decision_unix":1779712852.2555609,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":19456,"affinity_cache_ratio":0.9869128538094755,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":19456,"estimated_new_tokens":258,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":0,"new_prefill":19714,"score_linear":68259.0,"score_lmetric":19714},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":0,"new_prefill":19714,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":19714,"score_linear":150402.0,"score_lmetric":284936},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":19714,"score_linear":76969.0,"score_lmetric":96683},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":19714,"score_linear":199977.0,"score_lmetric":420993},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":19714,"score_linear":62685.0,"score_lmetric":82399},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":19714,"score_linear":80246.0,"score_lmetric":19714},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":58758,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1297,"cache_hit":19456,"new_prefill":258,"score_linear":39302.0,"score_lmetric":258}],"chosen_score_linear":39302.0,"chosen_score_lmetric":258,"t_first_token":281987.296856266,"t_first_token_unix":1779712852.3107994,"t_done":281988.475594613,"t_done_unix":1779712853.489538},{"request_id":"1355951:2:1363365:540","session_id":"1355951","input_length":37232,"t_proxy_recv":281987.764878716,"t_decision_unix":1779712852.77882,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9488611946712505,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":35328,"estimated_new_tokens":1904,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":0,"new_prefill":37232,"score_linear":68259.0,"score_lmetric":37232},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":0,"new_prefill":37232,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":37232,"score_linear":150402.0,"score_lmetric":319972},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":37232,"score_linear":76969.0,"score_lmetric":114201},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":37232,"score_linear":199977.0,"score_lmetric":473547},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":37232,"score_linear":62685.0,"score_lmetric":99917},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":37232,"score_linear":80246.0,"score_lmetric":37232},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":78472,"ongoing_decode_tokens":78472,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1297,"cache_hit":35328,"new_prefill":1904,"score_linear":43144.0,"score_lmetric":3808}],"chosen_score_linear":43144.0,"chosen_score_lmetric":3808,"t_first_token":281988.362741297,"t_first_token_unix":1779712853.376685,"t_done":281989.387187066,"t_done_unix":1779712854.4011304},{"request_id":"1363943:1:1363943:543","session_id":"1363943","input_length":12636,"t_proxy_recv":281989.736749376,"t_decision_unix":1779712854.7506907,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":12636,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":8704,"new_prefill":3932,"score_linear":59555.0,"score_lmetric":3932},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":12636,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":12636,"score_linear":150402.0,"score_lmetric":270780},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":4096,"new_prefill":8540,"score_linear":72873.0,"score_lmetric":85509},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":12636,"score_linear":199977.0,"score_lmetric":399759},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":12636,"score_linear":62685.0,"score_lmetric":75321},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":4096,"new_prefill":8540,"score_linear":76150.0,"score_lmetric":8540},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":58758,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":4096,"new_prefill":8540,"score_linear":54662.0,"score_lmetric":8540}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281990.786988758,"t_first_token_unix":1779712855.800932,"t_done":281992.152654417,"t_done_unix":1779712857.1665974},{"request_id":"1364128:1:1364128:545","session_id":"1364128","input_length":8706,"t_proxy_recv":281990.221725995,"t_decision_unix":1779712855.2356672,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":8706,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8706,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":0,"new_prefill":8706,"score_linear":68259.0,"score_lmetric":8706},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":12636,"ongoing_decode_tokens":0,"pending_prefill_tokens":12636,"num_requests":1,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":6656,"new_prefill":2050,"score_linear":5980.0,"score_lmetric":14686},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":8706,"score_linear":150402.0,"score_lmetric":262920},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":8706,"score_linear":76969.0,"score_lmetric":85675},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":8706,"score_linear":199977.0,"score_lmetric":387969},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":8706,"score_linear":62685.0,"score_lmetric":71391},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":94143,"ongoing_decode_tokens":80246,"pending_prefill_tokens":2633,"num_requests":2,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":8706,"score_linear":94143.0,"score_lmetric":22678},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":58758,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":8706,"score_linear":58758.0,"score_lmetric":8706}],"chosen_score_linear":68259.0,"chosen_score_lmetric":8706,"t_first_token":281990.980877183,"t_first_token_unix":1779712855.9948206,"t_done":281992.496289412,"t_done_unix":1779712857.5102334},{"request_id":"1279412:24:1364169:546","session_id":"1279412","input_length":98038,"t_proxy_recv":281990.363941714,"t_decision_unix":1779712855.3778832,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":97792,"affinity_cache_ratio":0.9974907688855342,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":97792,"estimated_new_tokens":246,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":76965,"ongoing_decode_tokens":68259,"pending_prefill_tokens":8706,"num_requests":2,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":0,"new_prefill":98038,"score_linear":76965.0,"score_lmetric":213488},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":12636,"ongoing_decode_tokens":0,"pending_prefill_tokens":12636,"num_requests":1,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":98038,"score_linear":12636.0,"score_lmetric":110674},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":98038,"score_linear":150402.0,"score_lmetric":441584},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":98038,"score_linear":76969.0,"score_lmetric":175007},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":98038,"score_linear":199977.0,"score_lmetric":655965},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":98038,"score_linear":62685.0,"score_lmetric":160723},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":94143,"ongoing_decode_tokens":80246,"pending_prefill_tokens":2633,"num_requests":2,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":98038,"score_linear":94143.0,"score_lmetric":201342},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":58758,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":97792,"new_prefill":246,"score_linear":-39034.0,"score_lmetric":246}],"chosen_score_linear":-39034.0,"chosen_score_lmetric":246,"t_first_token":281990.580697914,"t_first_token_unix":1779712855.5946412,"t_done":281993.452411474,"t_done_unix":1779712858.4663556},{"request_id":"1364960:1:1364960:549","session_id":"1364960","input_length":2019,"t_proxy_recv":281993.145810879,"t_decision_unix":1779712858.1597521,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":2019,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1055,"cache_hit":0,"new_prefill":2019,"score_linear":68259.0,"score_lmetric":2019},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1064,"cache_hit":0,"new_prefill":2019,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":2019,"score_linear":150402.0,"score_lmetric":249546},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":2019,"score_linear":76969.0,"score_lmetric":78988},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":2019,"score_linear":199977.0,"score_lmetric":367908},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":2019,"score_linear":62685.0,"score_lmetric":64704},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":94143,"ongoing_decode_tokens":94143,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":2019,"score_linear":94143.0,"score_lmetric":4038},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":176602,"ongoing_decode_tokens":176602,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":2019,"score_linear":176602.0,"score_lmetric":6057}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281993.274289944,"t_first_token_unix":1779712858.2882333,"t_done":281993.453998249,"t_done_unix":1779712858.4679534},{"request_id":"1313181:2:1358046:519","session_id":"1313181","input_length":58758,"t_proxy_recv":281968.82682345,"t_decision_unix":1779712833.8407648,"policy":"unified","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":24064,"affinity_cache_ratio":0.40954423227475406,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":34694,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":0,"new_prefill":58758,"score_linear":68259.0,"score_lmetric":58758},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":48193,"ongoing_decode_tokens":48193,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":982,"cache_hit":0,"new_prefill":58758,"score_linear":48193.0,"score_lmetric":58758},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":58758,"score_linear":150402.0,"score_lmetric":363024},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1321,"cache_hit":0,"new_prefill":58758,"score_linear":21371.0,"score_lmetric":58758},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":58758,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":58758,"score_linear":83526.0,"score_lmetric":142284},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":58758,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":24064,"new_prefill":34694,"score_linear":-24064.0,"score_lmetric":0}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":281980.253474002,"t_first_token_unix":1779712845.2674224,"t_done":281994.080032842,"t_done_unix":1779712859.093976},{"request_id":"1363440:2:1365095:551","session_id":"1363440","input_length":3866,"t_proxy_recv":281993.728315107,"t_decision_unix":1779712858.7422564,"policy":"unified","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":512,"affinity_cache_ratio":0.13243662700465597,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":512,"estimated_new_tokens":3354,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1055,"cache_hit":0,"new_prefill":3866,"score_linear":68259.0,"score_lmetric":3866},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1068,"cache_hit":512,"new_prefill":3354,"score_linear":-512.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":3866,"score_linear":150402.0,"score_lmetric":253240},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":91690,"ongoing_decode_tokens":0,"pending_prefill_tokens":78378,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":3866,"score_linear":91690.0,"score_lmetric":164488},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":3866,"score_linear":199977.0,"score_lmetric":373449},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":3866,"score_linear":62685.0,"score_lmetric":66551},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":94143,"ongoing_decode_tokens":94143,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":3866,"score_linear":94143.0,"score_lmetric":7732},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":78564,"ongoing_decode_tokens":78564,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":3866,"score_linear":78564.0,"score_lmetric":7732}],"chosen_score_linear":-512.0,"chosen_score_lmetric":0,"t_first_token":281993.941183349,"t_first_token_unix":1779712858.9551265,"t_done":281994.197238631,"t_done_unix":1779712859.2111819},{"request_id":"1355484:4:1364784:548","session_id":"1355484","input_length":19806,"t_proxy_recv":281992.534072058,"t_decision_unix":1779712857.5480137,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":19456,"affinity_cache_ratio":0.9823285872967787,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":19456,"estimated_new_tokens":350,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1055,"cache_hit":0,"new_prefill":19806,"score_linear":68259.0,"score_lmetric":19806},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1064,"cache_hit":0,"new_prefill":19806,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":19806,"score_linear":150402.0,"score_lmetric":285120},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":19806,"score_linear":76969.0,"score_lmetric":96775},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":19806,"score_linear":199977.0,"score_lmetric":421269},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":19806,"score_linear":62685.0,"score_lmetric":82491},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":94143,"ongoing_decode_tokens":94143,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":19806,"score_linear":94143.0,"score_lmetric":39612},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":156796,"ongoing_decode_tokens":156796,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":19456,"new_prefill":350,"score_linear":137340.0,"score_lmetric":700}],"chosen_score_linear":137340.0,"chosen_score_lmetric":700,"t_first_token":281992.622751008,"t_first_token_unix":1779712857.6366942,"t_done":281994.657385558,"t_done_unix":1779712859.671329},{"request_id":"1313181:3:1316742:330","session_id":"1313181","input_length":24451,"t_proxy_recv":281994.085872027,"t_decision_unix":1779712859.0998137,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9841724264856243,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":387,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1055,"cache_hit":0,"new_prefill":24451,"score_linear":68259.0,"score_lmetric":24451},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":3866,"ongoing_decode_tokens":3866,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1068,"cache_hit":0,"new_prefill":24451,"score_linear":3866.0,"score_lmetric":24451},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":24451,"score_linear":150402.0,"score_lmetric":294410},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":91690,"ongoing_decode_tokens":0,"pending_prefill_tokens":78378,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":24451,"score_linear":91690.0,"score_lmetric":205658},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":24451,"score_linear":199977.0,"score_lmetric":435204},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":24451,"score_linear":62685.0,"score_lmetric":87136},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":94143,"ongoing_decode_tokens":94143,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":24451,"score_linear":94143.0,"score_lmetric":48902},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":19806,"ongoing_decode_tokens":19806,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1369,"cache_hit":24064,"new_prefill":387,"score_linear":-4258.0,"score_lmetric":387}],"chosen_score_linear":-4258.0,"chosen_score_lmetric":387,"t_first_token":281994.132364948,"t_first_token_unix":1779712859.1463082,"t_done":281994.842883699,"t_done_unix":1779712859.8568275},{"request_id":"1365440:1:1365440:552","session_id":"1365440","input_length":9913,"t_proxy_recv":281994.949499901,"t_decision_unix":1779712859.963433,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":9913,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1055,"cache_hit":0,"new_prefill":9913,"score_linear":68259.0,"score_lmetric":9913},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":9913,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":9913,"score_linear":150402.0,"score_lmetric":265334},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":91690,"ongoing_decode_tokens":0,"pending_prefill_tokens":78378,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":9913,"score_linear":91690.0,"score_lmetric":176582},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":9913,"score_linear":199977.0,"score_lmetric":391590},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":9913,"score_linear":62685.0,"score_lmetric":72598},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":94143,"ongoing_decode_tokens":94143,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":9913,"score_linear":94143.0,"score_lmetric":19826},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":59514,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":1,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":9913,"score_linear":59514.0,"score_lmetric":45363}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281995.708683289,"t_first_token_unix":1779712860.722627,"t_done":281995.822872303,"t_done_unix":1779712860.8368158},{"request_id":"1364090:1:1364090:544","session_id":"1364090","input_length":13897,"t_proxy_recv":281990.115222934,"t_decision_unix":1779712855.1291642,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":2633,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":2633,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":7680,"new_prefill":6217,"score_linear":60579.0,"score_lmetric":6217},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":12636,"ongoing_decode_tokens":0,"pending_prefill_tokens":12636,"num_requests":1,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":13897,"score_linear":12636.0,"score_lmetric":26533},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":13897,"score_linear":150402.0,"score_lmetric":273302},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":7680,"new_prefill":6217,"score_linear":69289.0,"score_lmetric":83186},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":13897,"score_linear":199977.0,"score_lmetric":403542},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":13897,"score_linear":62685.0,"score_lmetric":76582},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":11264,"new_prefill":2633,"score_linear":68982.0,"score_lmetric":2633},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":58758,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":7680,"new_prefill":6217,"score_linear":51078.0,"score_lmetric":6217}],"chosen_score_linear":68982.0,"chosen_score_lmetric":2633,"t_first_token":281990.474507036,"t_first_token_unix":1779712855.4884503,"t_done":281996.945861748,"t_done_unix":1779712861.9598053},{"request_id":"1366128:1:1366128:553","session_id":"1366128","input_length":11154,"t_proxy_recv":281997.243078796,"t_decision_unix":1779712862.2570202,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":11154,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1055,"cache_hit":0,"new_prefill":11154,"score_linear":68259.0,"score_lmetric":11154},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1093,"cache_hit":0,"new_prefill":11154,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":11154,"score_linear":150402.0,"score_lmetric":267816},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":91690,"ongoing_decode_tokens":0,"pending_prefill_tokens":78378,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":11154,"score_linear":91690.0,"score_lmetric":179064},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":11154,"score_linear":199977.0,"score_lmetric":395313},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":11154,"score_linear":62685.0,"score_lmetric":73839},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":11154,"score_linear":80246.0,"score_lmetric":11154},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":59514,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":1,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":11154,"score_linear":59514.0,"score_lmetric":46604}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":281998.128319364,"t_first_token_unix":1779712863.142263,"t_done":281998.717519058,"t_done_unix":1779712863.7314622},{"request_id":"1344773:11:1366387:555","session_id":"1344773","input_length":35071,"t_proxy_recv":281998.216674765,"t_decision_unix":1779712863.230616,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9927290353853612,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":255,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1055,"cache_hit":34816,"new_prefill":255,"score_linear":33443.0,"score_lmetric":255},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":11154,"ongoing_decode_tokens":11154,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1093,"cache_hit":0,"new_prefill":35071,"score_linear":11154.0,"score_lmetric":35071},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":35071,"score_linear":150402.0,"score_lmetric":315650},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":91690,"ongoing_decode_tokens":0,"pending_prefill_tokens":78378,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":35071,"score_linear":91690.0,"score_lmetric":226898},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":35071,"score_linear":199977.0,"score_lmetric":467064},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":35071,"score_linear":62685.0,"score_lmetric":97756},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":35071,"score_linear":80246.0,"score_lmetric":35071},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":157781,"ongoing_decode_tokens":0,"pending_prefill_tokens":35925,"num_requests":2,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":35071,"score_linear":157781.0,"score_lmetric":141992}],"chosen_score_linear":33443.0,"chosen_score_lmetric":255,"t_first_token":281998.30956215,"t_first_token_unix":1779712863.3235054,"t_done":281999.323025952,"t_done_unix":1779712864.3369694},{"request_id":"1363943:2:1366428:556","session_id":"1363943","input_length":12984,"t_proxy_recv":281998.352482864,"t_decision_unix":1779712863.3664243,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":12800,"affinity_cache_ratio":0.9858287122612446,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":184,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":103330,"ongoing_decode_tokens":103330,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1055,"cache_hit":8704,"new_prefill":4280,"score_linear":94626.0,"score_lmetric":8560},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":11154,"ongoing_decode_tokens":11154,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1093,"cache_hit":12800,"new_prefill":184,"score_linear":-1646.0,"score_lmetric":184},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":12984,"score_linear":150402.0,"score_lmetric":271476},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":91690,"ongoing_decode_tokens":0,"pending_prefill_tokens":78378,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":4096,"new_prefill":8888,"score_linear":87594.0,"score_lmetric":174532},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":12984,"score_linear":199977.0,"score_lmetric":400803},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":12984,"score_linear":62685.0,"score_lmetric":75669},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":4096,"new_prefill":8888,"score_linear":76150.0,"score_lmetric":8888},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":157781,"ongoing_decode_tokens":0,"pending_prefill_tokens":35925,"num_requests":2,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":4096,"new_prefill":8888,"score_linear":153685.0,"score_lmetric":89626}],"chosen_score_linear":-1646.0,"chosen_score_lmetric":184,"t_first_token":281998.410052598,"t_first_token_unix":1779712863.423996,"t_done":282000.281439641,"t_done_unix":1779712865.2953827},{"request_id":"1314357:4:1352612:497","session_id":"1314357","input_length":68259,"t_proxy_recv":281949.895115731,"t_decision_unix":1779712814.9090571,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":67072,"affinity_cache_ratio":0.9826103517484874,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":67072,"estimated_new_tokens":1187,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55015,"ongoing_decode_tokens":0,"pending_prefill_tokens":9447,"num_requests":2,"active_p_offloads":0,"cached_blocks":1015,"cache_hit":67072,"new_prefill":1187,"score_linear":-12057.0,"score_lmetric":21268},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":68259,"score_linear":62618.0,"score_lmetric":68259},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1248,"cache_hit":0,"new_prefill":68259,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":68259,"score_linear":21371.0,"score_lmetric":68259},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18840,"ongoing_decode_tokens":0,"pending_prefill_tokens":18840,"num_requests":1,"active_p_offloads":0,"cached_blocks":1264,"cache_hit":0,"new_prefill":68259,"score_linear":18840.0,"score_lmetric":87099},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":10575,"ongoing_decode_tokens":0,"pending_prefill_tokens":10575,"num_requests":1,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":68259,"score_linear":10575.0,"score_lmetric":78834},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1197,"cache_hit":0,"new_prefill":68259,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":68259,"score_linear":96026.0,"score_lmetric":68259}],"chosen_score_linear":-12057.0,"chosen_score_lmetric":21268,"t_first_token":281950.424656059,"t_first_token_unix":1779712815.4385993,"t_done":282000.989481263,"t_done_unix":1779712866.0034244},{"request_id":"1363093:1:1363093:538","session_id":"1363093","input_length":62685,"t_proxy_recv":281986.738171897,"t_decision_unix":1779712851.7521133,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":62685,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":0,"new_prefill":62685,"score_linear":68259.0,"score_lmetric":62685},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":0,"new_prefill":62685,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":62685,"score_linear":150402.0,"score_lmetric":370878},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":62685,"score_linear":76969.0,"score_lmetric":139654},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":183920,"ongoing_decode_tokens":0,"pending_prefill_tokens":118384,"num_requests":2,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":62685,"score_linear":183920.0,"score_lmetric":362138},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":62685,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":62685,"score_linear":80246.0,"score_lmetric":62685},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":78251,"ongoing_decode_tokens":78251,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1297,"cache_hit":0,"new_prefill":62685,"score_linear":78251.0,"score_lmetric":125370}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282000.682092332,"t_first_token_unix":1779712865.6960356,"t_done":282001.21329111,"t_done_unix":1779712866.2272344},{"request_id":"1367424:1:1367424:569","session_id":"1367424","input_length":3392,"t_proxy_recv":282001.978958572,"t_decision_unix":1779712866.9929001,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.5,"fallback_score":17141,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":3392,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":58578,"ongoing_decode_tokens":0,"pending_prefill_tokens":49874,"num_requests":2,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":3392,"score_linear":58578.0,"score_lmetric":106532},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":44360,"ongoing_decode_tokens":0,"pending_prefill_tokens":24904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1114,"cache_hit":0,"new_prefill":3392,"score_linear":44360.0,"score_lmetric":84888},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":3392,"score_linear":150402.0,"score_lmetric":252292},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":127991,"ongoing_decode_tokens":0,"pending_prefill_tokens":80887,"num_requests":3,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":3392,"score_linear":127991.0,"score_lmetric":252837},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":3392,"score_linear":199977.0,"score_lmetric":372027},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13749,"ongoing_decode_tokens":0,"pending_prefill_tokens":13749,"num_requests":1,"active_p_offloads":0,"cached_blocks":1535,"cache_hit":0,"new_prefill":3392,"score_linear":13749.0,"score_lmetric":17141},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":107083,"ongoing_decode_tokens":80246,"pending_prefill_tokens":26837,"num_requests":2,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":3392,"score_linear":107083.0,"score_lmetric":60458},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":218824,"ongoing_decode_tokens":0,"pending_prefill_tokens":43720,"num_requests":4,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":3392,"score_linear":218824.0,"score_lmetric":188448}],"chosen_score_linear":13749.0,"chosen_score_lmetric":17141,"t_first_token":282002.939197405,"t_first_token_unix":1779712867.9531407,"t_done":282003.009170231,"t_done_unix":1779712868.0231133},{"request_id":"1367312:1:1367312:566","session_id":"1367312","input_length":13749,"t_proxy_recv":282001.575100165,"t_decision_unix":1779712866.5890412,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":13749,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":58578,"ongoing_decode_tokens":0,"pending_prefill_tokens":49874,"num_requests":2,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":13749,"score_linear":58578.0,"score_lmetric":127246},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":34226,"ongoing_decode_tokens":0,"pending_prefill_tokens":23474,"num_requests":2,"active_p_offloads":0,"cached_blocks":1114,"cache_hit":0,"new_prefill":13749,"score_linear":34226.0,"score_lmetric":74446},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":13749,"score_linear":150402.0,"score_lmetric":273006},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":127991,"ongoing_decode_tokens":0,"pending_prefill_tokens":80887,"num_requests":3,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":13749,"score_linear":127991.0,"score_lmetric":283908},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":13749,"score_linear":199977.0,"score_lmetric":403098},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1535,"cache_hit":0,"new_prefill":13749,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":13749,"score_linear":80246.0,"score_lmetric":13749},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":218824,"ongoing_decode_tokens":0,"pending_prefill_tokens":43720,"num_requests":4,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":13749,"score_linear":218824.0,"score_lmetric":229876}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282002.745277406,"t_first_token_unix":1779712867.759221,"t_done":282003.276761254,"t_done_unix":1779712868.2907045},{"request_id":"1362265:2:1367373:568","session_id":"1362265","input_length":10134,"t_proxy_recv":282001.779747057,"t_decision_unix":1779712866.7936883,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":8704,"affinity_cache_ratio":0.8588908624432603,"affinity_num_requests":2,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"cache_hit":8704,"estimated_new_tokens":1430,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":58578,"ongoing_decode_tokens":0,"pending_prefill_tokens":49874,"num_requests":2,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":6656,"new_prefill":3478,"score_linear":51922.0,"score_lmetric":106704},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":34226,"ongoing_decode_tokens":0,"pending_prefill_tokens":23474,"num_requests":2,"active_p_offloads":0,"cached_blocks":1114,"cache_hit":8704,"new_prefill":1430,"score_linear":25522.0,"score_lmetric":49808},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":10134,"score_linear":150402.0,"score_lmetric":265776},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":127991,"ongoing_decode_tokens":0,"pending_prefill_tokens":80887,"num_requests":3,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":10134,"score_linear":127991.0,"score_lmetric":273063},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":10134,"score_linear":199977.0,"score_lmetric":392253},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13749,"ongoing_decode_tokens":0,"pending_prefill_tokens":13749,"num_requests":1,"active_p_offloads":0,"cached_blocks":1535,"cache_hit":0,"new_prefill":10134,"score_linear":13749.0,"score_lmetric":23883},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":107083,"ongoing_decode_tokens":80246,"pending_prefill_tokens":26837,"num_requests":2,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":10134,"score_linear":107083.0,"score_lmetric":73942},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":218824,"ongoing_decode_tokens":0,"pending_prefill_tokens":43720,"num_requests":4,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":10134,"score_linear":218824.0,"score_lmetric":215416}],"chosen_score_linear":25522.0,"chosen_score_lmetric":49808,"t_first_token":282003.590965392,"t_first_token_unix":1779712868.6049085,"t_done":282003.88797247,"t_done_unix":1779712868.9019153},{"request_id":"1366128:2:1367220:563","session_id":"1366128","input_length":15411,"t_proxy_recv":282001.278373554,"t_decision_unix":1779712866.2923152,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":10752,"affinity_cache_ratio":0.6976834728440724,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":10752,"estimated_new_tokens":4659,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":15411,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":18815,"ongoing_decode_tokens":0,"pending_prefill_tokens":18815,"num_requests":1,"active_p_offloads":0,"cached_blocks":1114,"cache_hit":10752,"new_prefill":4659,"score_linear":8063.0,"score_lmetric":23474},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":15411,"score_linear":150402.0,"score_lmetric":276330},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":127991,"ongoing_decode_tokens":0,"pending_prefill_tokens":80887,"num_requests":3,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":15411,"score_linear":127991.0,"score_lmetric":288894},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":15411,"score_linear":199977.0,"score_lmetric":408084},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1535,"cache_hit":0,"new_prefill":15411,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":15411,"score_linear":80246.0,"score_lmetric":15411},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":218824,"ongoing_decode_tokens":0,"pending_prefill_tokens":43720,"num_requests":4,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":15411,"score_linear":218824.0,"score_lmetric":236524}],"chosen_score_linear":8063.0,"chosen_score_lmetric":23474,"t_first_token":282003.512051082,"t_first_token_unix":1779712868.525994,"t_done":282004.303704526,"t_done_unix":1779712869.3176477},{"request_id":"1367329:1:1367329:567","session_id":"1367329","input_length":26837,"t_proxy_recv":282001.638157651,"t_decision_unix":1779712866.6520987,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.25,"fallback_score":26837,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":26837,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":58578,"ongoing_decode_tokens":0,"pending_prefill_tokens":49874,"num_requests":2,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":26837,"score_linear":58578.0,"score_lmetric":153422},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":34226,"ongoing_decode_tokens":0,"pending_prefill_tokens":23474,"num_requests":2,"active_p_offloads":0,"cached_blocks":1114,"cache_hit":0,"new_prefill":26837,"score_linear":34226.0,"score_lmetric":100622},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":26837,"score_linear":150402.0,"score_lmetric":299182},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":127991,"ongoing_decode_tokens":0,"pending_prefill_tokens":80887,"num_requests":3,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":26837,"score_linear":127991.0,"score_lmetric":323172},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":26837,"score_linear":199977.0,"score_lmetric":442362},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13749,"ongoing_decode_tokens":0,"pending_prefill_tokens":13749,"num_requests":1,"active_p_offloads":0,"cached_blocks":1535,"cache_hit":0,"new_prefill":26837,"score_linear":13749.0,"score_lmetric":40586},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":26837,"score_linear":80246.0,"score_lmetric":26837},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":218824,"ongoing_decode_tokens":0,"pending_prefill_tokens":43720,"num_requests":4,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":26837,"score_linear":218824.0,"score_lmetric":282228}],"chosen_score_linear":80246.0,"chosen_score_lmetric":26837,"t_first_token":282004.956327956,"t_first_token_unix":1779712869.9702713,"t_done":282006.076914383,"t_done_unix":1779712871.0908575},{"request_id":"1368070:1:1368070:575","session_id":"1368070","input_length":21773,"t_proxy_recv":282004.295834089,"t_decision_unix":1779712869.3097754,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.625,"fallback_score":43546,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":21773,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93772,"ongoing_decode_tokens":0,"pending_prefill_tokens":50252,"num_requests":3,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":21773,"score_linear":93772.0,"score_lmetric":216075},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":34226,"ongoing_decode_tokens":34226,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1116,"cache_hit":0,"new_prefill":21773,"score_linear":34226.0,"score_lmetric":43546},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":21773,"score_linear":150402.0,"score_lmetric":289054},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":127991,"ongoing_decode_tokens":91690,"pending_prefill_tokens":2509,"num_requests":3,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":21773,"score_linear":127991.0,"score_lmetric":72846},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":21773,"score_linear":199977.0,"score_lmetric":427170},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":88420,"ongoing_decode_tokens":0,"pending_prefill_tokens":75108,"num_requests":2,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":21773,"score_linear":88420.0,"score_lmetric":193762},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":107083,"ongoing_decode_tokens":80246,"pending_prefill_tokens":26837,"num_requests":2,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":21773,"score_linear":107083.0,"score_lmetric":97220},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":218824,"ongoing_decode_tokens":0,"pending_prefill_tokens":43720,"num_requests":4,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":21773,"score_linear":218824.0,"score_lmetric":261972}],"chosen_score_linear":34226.0,"chosen_score_lmetric":43546,"t_first_token":282006.694732223,"t_first_token_unix":1779712871.7086754,"t_done":282007.231588761,"t_done_unix":1779712872.2455318},{"request_id":"1364960:2:1368239:578","session_id":"1364960","input_length":7048,"t_proxy_recv":282004.871040372,"t_decision_unix":1779712869.8849819,"policy":"unified","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":2048,"affinity_cache_ratio":0.2905788876276958,"affinity_num_requests":2,"avg_num_requests":2.75,"fallback_score":53546,"tie_break_used":false,"cache_hit":2048,"estimated_new_tokens":5000,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93772,"ongoing_decode_tokens":0,"pending_prefill_tokens":50252,"num_requests":3,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":7048,"score_linear":93772.0,"score_lmetric":171900},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":40588,"ongoing_decode_tokens":18815,"pending_prefill_tokens":21773,"num_requests":2,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":2048,"new_prefill":5000,"score_linear":38540.0,"score_lmetric":53546},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":7048,"score_linear":150402.0,"score_lmetric":259604},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":136836,"ongoing_decode_tokens":91690,"pending_prefill_tokens":11354,"num_requests":4,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":7048,"score_linear":136836.0,"score_lmetric":73608},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":7048,"score_linear":199977.0,"score_lmetric":382995},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":88420,"ongoing_decode_tokens":0,"pending_prefill_tokens":75108,"num_requests":2,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":7048,"score_linear":88420.0,"score_lmetric":164312},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":107083,"ongoing_decode_tokens":80246,"pending_prefill_tokens":26837,"num_requests":2,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":7048,"score_linear":107083.0,"score_lmetric":67770},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":218824,"ongoing_decode_tokens":0,"pending_prefill_tokens":43720,"num_requests":4,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":7048,"score_linear":218824.0,"score_lmetric":203072}],"chosen_score_linear":38540.0,"chosen_score_lmetric":53546,"t_first_token":282007.017397936,"t_first_token_unix":1779712872.031341,"t_done":282007.466717258,"t_done_unix":1779712872.4806602},{"request_id":"1355484:5:1366753:560","session_id":"1355484","input_length":20026,"t_proxy_recv":281999.41715998,"t_decision_unix":1779712864.4311013,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":19968,"affinity_cache_ratio":0.997103765105363,"affinity_num_requests":3,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":58,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1055,"cache_hit":0,"new_prefill":20026,"score_linear":68259.0,"score_lmetric":20026},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":12984,"ongoing_decode_tokens":12984,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1114,"cache_hit":0,"new_prefill":20026,"score_linear":12984.0,"score_lmetric":20026},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":20026,"score_linear":150402.0,"score_lmetric":285560},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":127991,"ongoing_decode_tokens":0,"pending_prefill_tokens":80887,"num_requests":3,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":20026,"score_linear":127991.0,"score_lmetric":302739},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":20026,"score_linear":199977.0,"score_lmetric":421929},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":20026,"score_linear":62685.0,"score_lmetric":82711},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":20026,"score_linear":80246.0,"score_lmetric":20026},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":198798,"ongoing_decode_tokens":0,"pending_prefill_tokens":43662,"num_requests":3,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":19968,"new_prefill":58,"score_linear":178830.0,"score_lmetric":131160}],"chosen_score_linear":178830.0,"chosen_score_lmetric":131160,"t_first_token":282006.779837341,"t_first_token_unix":1779712871.7937808,"t_done":282007.831686654,"t_done_unix":1779712872.84563},{"request_id":"1367169:1:1367169:562","session_id":"1367169","input_length":18815,"t_proxy_recv":282001.091567137,"t_decision_unix":1779712866.1055086,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":18815,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":18815,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1114,"cache_hit":0,"new_prefill":18815,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":18815,"score_linear":150402.0,"score_lmetric":283138},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":127991,"ongoing_decode_tokens":0,"pending_prefill_tokens":80887,"num_requests":3,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":18815,"score_linear":127991.0,"score_lmetric":299106},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":18815,"score_linear":199977.0,"score_lmetric":418296},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":62685,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":18815,"score_linear":62685.0,"score_lmetric":18815},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":11776,"new_prefill":7039,"score_linear":68470.0,"score_lmetric":7039},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":218824,"ongoing_decode_tokens":0,"pending_prefill_tokens":43720,"num_requests":4,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":18815,"score_linear":218824.0,"score_lmetric":250140}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282003.510291293,"t_first_token_unix":1779712868.5242345,"t_done":282008.076180099,"t_done_unix":1779712873.0901234},{"request_id":"1361694:1:1361694:534","session_id":"1361694","input_length":76969,"t_proxy_recv":281981.993516993,"t_decision_unix":1779712847.0074584,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":76969,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":102938,"ongoing_decode_tokens":102938,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":0,"new_prefill":76969,"score_linear":102938.0,"score_lmetric":153938},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":16480,"ongoing_decode_tokens":16480,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":989,"cache_hit":0,"new_prefill":76969,"score_linear":16480.0,"score_lmetric":76969},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":76969,"score_linear":150402.0,"score_lmetric":399446},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":76969,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":183920,"ongoing_decode_tokens":0,"pending_prefill_tokens":118384,"num_requests":2,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":76969,"score_linear":183920.0,"score_lmetric":390706},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":76969,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":3584,"new_prefill":73385,"score_linear":76662.0,"score_lmetric":73385},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":111729,"ongoing_decode_tokens":78251,"pending_prefill_tokens":5318,"num_requests":3,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":0,"new_prefill":76969,"score_linear":111729.0,"score_lmetric":246861}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282002.287096445,"t_first_token_unix":1779712867.3010397,"t_done":282008.397185958,"t_done_unix":1779712873.4111328},{"request_id":"1277428:5:1366630:559","session_id":"1277428","input_length":36301,"t_proxy_recv":281999.041540562,"t_decision_unix":1779712864.055482,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9308834467370045,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":33792,"estimated_new_tokens":2509,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":103330,"ongoing_decode_tokens":103330,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1055,"cache_hit":0,"new_prefill":36301,"score_linear":103330.0,"score_lmetric":72602},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":12984,"ongoing_decode_tokens":12984,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1114,"cache_hit":4608,"new_prefill":31693,"score_linear":8376.0,"score_lmetric":31693},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":36301,"score_linear":150402.0,"score_lmetric":318110},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":91690,"ongoing_decode_tokens":0,"pending_prefill_tokens":78378,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":33792,"new_prefill":2509,"score_linear":57898.0,"score_lmetric":161774},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":36301,"score_linear":199977.0,"score_lmetric":470754},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":36301,"score_linear":62685.0,"score_lmetric":98986},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":36301,"score_linear":80246.0,"score_lmetric":36301},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":198798,"ongoing_decode_tokens":0,"pending_prefill_tokens":43662,"num_requests":3,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":36301,"score_linear":198798.0,"score_lmetric":239889}],"chosen_score_linear":57898.0,"chosen_score_lmetric":161774,"t_first_token":282006.009366005,"t_first_token_unix":1779712871.0233092,"t_done":282008.493427072,"t_done_unix":1779712873.5073705},{"request_id":"1357132:2:1364990:550","session_id":"1357132","input_length":14721,"t_proxy_recv":281993.221435982,"t_decision_unix":1779712858.2353773,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":13312,"affinity_cache_ratio":0.9042863935873922,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":13312,"estimated_new_tokens":1409,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1055,"cache_hit":0,"new_prefill":14721,"score_linear":68259.0,"score_lmetric":14721},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":2019,"ongoing_decode_tokens":0,"pending_prefill_tokens":2019,"num_requests":1,"active_p_offloads":0,"cached_blocks":1064,"cache_hit":0,"new_prefill":14721,"score_linear":2019.0,"score_lmetric":16740},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":14721,"score_linear":150402.0,"score_lmetric":274950},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":13312,"new_prefill":1409,"score_linear":63657.0,"score_lmetric":78378},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":14721,"score_linear":199977.0,"score_lmetric":406014},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":14721,"score_linear":62685.0,"score_lmetric":77406},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":94143,"ongoing_decode_tokens":94143,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":14721,"score_linear":94143.0,"score_lmetric":29442},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":176602,"ongoing_decode_tokens":176602,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":14721,"score_linear":176602.0,"score_lmetric":44163}],"chosen_score_linear":63657.0,"chosen_score_lmetric":78378,"t_first_token":282002.287758918,"t_first_token_unix":1779712867.3017018,"t_done":282008.650363896,"t_done_unix":1779712873.664307},{"request_id":"1369172:1:1369172:581","session_id":"1369172","input_length":6345,"t_proxy_recv":282008.13848726,"t_decision_unix":1779712873.1524289,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":6345,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93772,"ongoing_decode_tokens":0,"pending_prefill_tokens":50252,"num_requests":3,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":6345,"score_linear":93772.0,"score_lmetric":169791},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1213,"cache_hit":0,"new_prefill":6345,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":6345,"score_linear":150402.0,"score_lmetric":258198},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":136836,"ongoing_decode_tokens":136836,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":6345,"score_linear":136836.0,"score_lmetric":25380},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":6345,"score_linear":199977.0,"score_lmetric":380886},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":6345,"score_linear":151655.0,"score_lmetric":246672},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":0,"new_prefill":6345,"score_linear":80246.0,"score_lmetric":6345},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":198798,"ongoing_decode_tokens":198798,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":6345,"score_linear":198798.0,"score_lmetric":19035}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282008.557870076,"t_first_token_unix":1779712873.571814,"t_done":282008.801968398,"t_done_unix":1779712873.8159122},{"request_id":"1357132:3:1366582:557","session_id":"1357132","input_length":14816,"t_proxy_recv":282008.655920027,"t_decision_unix":1779712873.6698616,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":14336,"affinity_cache_ratio":0.9676025917926566,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":480,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93772,"ongoing_decode_tokens":0,"pending_prefill_tokens":50252,"num_requests":3,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":14816,"score_linear":93772.0,"score_lmetric":195204},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":6345,"ongoing_decode_tokens":6345,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1213,"cache_hit":0,"new_prefill":14816,"score_linear":6345.0,"score_lmetric":14816},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":14816,"score_linear":150402.0,"score_lmetric":275140},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":8845,"ongoing_decode_tokens":8845,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":14336,"new_prefill":480,"score_linear":-5491.0,"score_lmetric":480},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":14816,"score_linear":199977.0,"score_lmetric":406299},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":14816,"score_linear":151655.0,"score_lmetric":272085},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":0,"new_prefill":14816,"score_linear":80246.0,"score_lmetric":14816},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":198798,"ongoing_decode_tokens":198798,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":14816,"score_linear":198798.0,"score_lmetric":44448}],"chosen_score_linear":-5491.0,"chosen_score_lmetric":480,"t_first_token":282008.696643112,"t_first_token_unix":1779712873.7105863,"t_done":282008.865164162,"t_done_unix":1779712873.8791072},{"request_id":"1357132:4:1367654:570","session_id":"1357132","input_length":14949,"t_proxy_recv":282008.869303581,"t_decision_unix":1779712873.883245,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":14336,"affinity_cache_ratio":0.9589939126362967,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":613,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93772,"ongoing_decode_tokens":0,"pending_prefill_tokens":50252,"num_requests":3,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":14949,"score_linear":93772.0,"score_lmetric":195603},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1225,"cache_hit":0,"new_prefill":14949,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":14949,"score_linear":150402.0,"score_lmetric":275406},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":8845,"ongoing_decode_tokens":8845,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":14336,"new_prefill":613,"score_linear":-5491.0,"score_lmetric":613},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":14949,"score_linear":199977.0,"score_lmetric":406698},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":14949,"score_linear":151655.0,"score_lmetric":272484},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":0,"new_prefill":14949,"score_linear":80246.0,"score_lmetric":14949},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":198798,"ongoing_decode_tokens":198798,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":14949,"score_linear":198798.0,"score_lmetric":44847}],"chosen_score_linear":-5491.0,"chosen_score_lmetric":613,"t_first_token":282008.936861265,"t_first_token_unix":1779712873.9508045,"t_done":282009.144254302,"t_done_unix":1779712874.1581976},{"request_id":"1357132:5:1368445:579","session_id":"1357132","input_length":15871,"t_proxy_recv":282009.149480789,"t_decision_unix":1779712874.163422,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":14848,"affinity_cache_ratio":0.93554281393737,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":14848,"estimated_new_tokens":1023,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93772,"ongoing_decode_tokens":0,"pending_prefill_tokens":50252,"num_requests":3,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":15871,"score_linear":93772.0,"score_lmetric":198369},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1225,"cache_hit":0,"new_prefill":15871,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":15871,"score_linear":150402.0,"score_lmetric":277250},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":8845,"ongoing_decode_tokens":8845,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1502,"cache_hit":14848,"new_prefill":1023,"score_linear":-6003.0,"score_lmetric":1023},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":15871,"score_linear":199977.0,"score_lmetric":409464},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":15871,"score_linear":151655.0,"score_lmetric":275250},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":0,"new_prefill":15871,"score_linear":80246.0,"score_lmetric":15871},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":198798,"ongoing_decode_tokens":198798,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":15871,"score_linear":198798.0,"score_lmetric":47613}],"chosen_score_linear":-6003.0,"chosen_score_lmetric":1023,"t_first_token":282009.31423134,"t_first_token_unix":1779712874.3281746,"t_done":282009.521422577,"t_done_unix":1779712874.535366},{"request_id":"1270340:4:1366608:558","session_id":"1270340","input_length":41017,"t_proxy_recv":281998.972108758,"t_decision_unix":1779712863.9860501,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":33280,"affinity_cache_ratio":0.8113708949947582,"affinity_num_requests":2,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":33280,"estimated_new_tokens":7737,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":103330,"ongoing_decode_tokens":103330,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1055,"cache_hit":0,"new_prefill":41017,"score_linear":103330.0,"score_lmetric":82034},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":12984,"ongoing_decode_tokens":12984,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1114,"cache_hit":0,"new_prefill":41017,"score_linear":12984.0,"score_lmetric":41017},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":41017,"score_linear":150402.0,"score_lmetric":327542},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":91690,"ongoing_decode_tokens":0,"pending_prefill_tokens":78378,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":41017,"score_linear":91690.0,"score_lmetric":238790},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":1536,"new_prefill":39481,"score_linear":198441.0,"score_lmetric":480294},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":41017,"score_linear":62685.0,"score_lmetric":103702},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":41017,"score_linear":80246.0,"score_lmetric":41017},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":157781,"ongoing_decode_tokens":0,"pending_prefill_tokens":35925,"num_requests":2,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":33280,"new_prefill":7737,"score_linear":124501.0,"score_lmetric":87324}],"chosen_score_linear":124501.0,"chosen_score_lmetric":87324,"t_first_token":282006.776315624,"t_first_token_unix":1779712871.790259,"t_done":282009.616480184,"t_done_unix":1779712874.6304235},{"request_id":"1368132:1:1368132:577","session_id":"1368132","input_length":8845,"t_proxy_recv":282004.541174155,"t_decision_unix":1779712869.5551155,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.625,"fallback_score":34062,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":8845,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93772,"ongoing_decode_tokens":0,"pending_prefill_tokens":50252,"num_requests":3,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":6656,"new_prefill":2189,"score_linear":87116.0,"score_lmetric":157323},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":40588,"ongoing_decode_tokens":18815,"pending_prefill_tokens":21773,"num_requests":2,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":6656,"new_prefill":2189,"score_linear":33932.0,"score_lmetric":47924},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":8845,"score_linear":150402.0,"score_lmetric":263198},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":127991,"ongoing_decode_tokens":91690,"pending_prefill_tokens":2509,"num_requests":3,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":8845,"score_linear":127991.0,"score_lmetric":34062},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":8845,"score_linear":199977.0,"score_lmetric":388386},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":88420,"ongoing_decode_tokens":0,"pending_prefill_tokens":75108,"num_requests":2,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":8845,"score_linear":88420.0,"score_lmetric":167906},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":107083,"ongoing_decode_tokens":80246,"pending_prefill_tokens":26837,"num_requests":2,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":8845,"score_linear":107083.0,"score_lmetric":71364},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":218824,"ongoing_decode_tokens":0,"pending_prefill_tokens":43720,"num_requests":4,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":8845,"score_linear":218824.0,"score_lmetric":210260}],"chosen_score_linear":127991.0,"chosen_score_lmetric":34062,"t_first_token":282006.735948883,"t_first_token_unix":1779712871.749892,"t_done":282009.954818944,"t_done_unix":1779712874.9687622},{"request_id":"1369768:1:1369768:584","session_id":"1369768","input_length":7712,"t_proxy_recv":282010.142076545,"t_decision_unix":1779712875.156018,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":7712,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93772,"ongoing_decode_tokens":0,"pending_prefill_tokens":50252,"num_requests":3,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":7712,"score_linear":93772.0,"score_lmetric":173892},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":18160,"ongoing_decode_tokens":18160,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1225,"cache_hit":0,"new_prefill":7712,"score_linear":18160.0,"score_lmetric":7712},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":7712,"score_linear":150402.0,"score_lmetric":260932},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1521,"cache_hit":0,"new_prefill":7712,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":7712,"score_linear":199977.0,"score_lmetric":384987},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":7712,"score_linear":151655.0,"score_lmetric":250773},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":0,"new_prefill":7712,"score_linear":80246.0,"score_lmetric":7712},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":177939,"ongoing_decode_tokens":177939,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":7712,"score_linear":177939.0,"score_lmetric":23136}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282010.681334347,"t_first_token_unix":1779712875.6952775,"t_done":282010.793263057,"t_done_unix":1779712875.8072062},{"request_id":"1364128:2:1367249:565","session_id":"1364128","input_length":9095,"t_proxy_recv":282001.372450513,"t_decision_unix":1779712866.3863919,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":8704,"affinity_cache_ratio":0.9570093457943926,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":8704,"estimated_new_tokens":391,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49483,"ongoing_decode_tokens":0,"pending_prefill_tokens":49483,"num_requests":1,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":8704,"new_prefill":391,"score_linear":40779.0,"score_lmetric":49874},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":34226,"ongoing_decode_tokens":0,"pending_prefill_tokens":23474,"num_requests":2,"active_p_offloads":0,"cached_blocks":1114,"cache_hit":6656,"new_prefill":2439,"score_linear":27570.0,"score_lmetric":51826},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":9095,"score_linear":150402.0,"score_lmetric":263698},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":127991,"ongoing_decode_tokens":0,"pending_prefill_tokens":80887,"num_requests":3,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":9095,"score_linear":127991.0,"score_lmetric":269946},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":9095,"score_linear":199977.0,"score_lmetric":389136},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1535,"cache_hit":0,"new_prefill":9095,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":9095,"score_linear":80246.0,"score_lmetric":9095},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":218824,"ongoing_decode_tokens":0,"pending_prefill_tokens":43720,"num_requests":4,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":9095,"score_linear":218824.0,"score_lmetric":211260}],"chosen_score_linear":40779.0,"chosen_score_lmetric":49874,"t_first_token":282010.682166071,"t_first_token_unix":1779712875.6961088,"t_done":282011.378024849,"t_done_unix":1779712876.3919683},{"request_id":"1355484:6:1369697:583","session_id":"1355484","input_length":20158,"t_proxy_recv":282009.923167304,"t_decision_unix":1779712874.9371083,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":19968,"affinity_cache_ratio":0.9905744617521579,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":190,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93772,"ongoing_decode_tokens":0,"pending_prefill_tokens":50252,"num_requests":3,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":20158,"score_linear":93772.0,"score_lmetric":211230},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":18160,"ongoing_decode_tokens":0,"pending_prefill_tokens":5360,"num_requests":1,"active_p_offloads":0,"cached_blocks":1225,"cache_hit":0,"new_prefill":20158,"score_linear":18160.0,"score_lmetric":25518},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":20158,"score_linear":150402.0,"score_lmetric":285824},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":8845,"ongoing_decode_tokens":8845,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1504,"cache_hit":0,"new_prefill":20158,"score_linear":8845.0,"score_lmetric":20158},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":20158,"score_linear":199977.0,"score_lmetric":422325},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":20158,"score_linear":151655.0,"score_lmetric":288111},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":0,"new_prefill":20158,"score_linear":80246.0,"score_lmetric":20158},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":157781,"ongoing_decode_tokens":157781,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":19968,"new_prefill":190,"score_linear":137813.0,"score_lmetric":380}],"chosen_score_linear":137813.0,"chosen_score_lmetric":380,"t_first_token":282010.017079494,"t_first_token_unix":1779712875.0310228,"t_done":282011.525843851,"t_done_unix":1779712876.5397873},{"request_id":"1344773:12:1367864:571","session_id":"1344773","input_length":35194,"t_proxy_recv":282003.47429794,"t_decision_unix":1779712868.488239,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9892595328749219,"affinity_num_requests":2,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":378,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":58578,"ongoing_decode_tokens":0,"pending_prefill_tokens":49874,"num_requests":2,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":34816,"new_prefill":378,"score_linear":23762.0,"score_lmetric":100504},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":44360,"ongoing_decode_tokens":0,"pending_prefill_tokens":24904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1114,"cache_hit":0,"new_prefill":35194,"score_linear":44360.0,"score_lmetric":180294},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":35194,"score_linear":150402.0,"score_lmetric":315896},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":127991,"ongoing_decode_tokens":91690,"pending_prefill_tokens":2509,"num_requests":3,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":35194,"score_linear":127991.0,"score_lmetric":113109},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":35194,"score_linear":199977.0,"score_lmetric":467433},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":35194,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":107083,"ongoing_decode_tokens":80246,"pending_prefill_tokens":26837,"num_requests":2,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":35194,"score_linear":107083.0,"score_lmetric":124062},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":218824,"ongoing_decode_tokens":0,"pending_prefill_tokens":43720,"num_requests":4,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":35194,"score_linear":218824.0,"score_lmetric":315656}],"chosen_score_linear":23762.0,"chosen_score_lmetric":100504,"t_first_token":282010.681515339,"t_first_token_unix":1779712875.695458,"t_done":282012.664875983,"t_done_unix":1779712877.6788187},{"request_id":"1344773:13:1370257:589","session_id":"1344773","input_length":35413,"t_proxy_recv":282012.672678911,"t_decision_unix":1779712877.6866205,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9975997515036851,"affinity_num_requests":2,"avg_num_requests":2.5,"fallback_score":null,"tie_break_used":false,"cache_hit":35328,"estimated_new_tokens":85,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57193,"ongoing_decode_tokens":49483,"pending_prefill_tokens":7710,"num_requests":2,"active_p_offloads":0,"cached_blocks":1060,"cache_hit":35328,"new_prefill":85,"score_linear":21865.0,"score_lmetric":15590},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":70968,"ongoing_decode_tokens":41467,"pending_prefill_tokens":7997,"num_requests":3,"active_p_offloads":0,"cached_blocks":1225,"cache_hit":0,"new_prefill":35413,"score_linear":70968.0,"score_lmetric":130230},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":35413,"score_linear":150402.0,"score_lmetric":316334},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":18301,"ongoing_decode_tokens":0,"pending_prefill_tokens":10621,"num_requests":1,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":35413,"score_linear":18301.0,"score_lmetric":46034},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":35413,"score_linear":199977.0,"score_lmetric":468090},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":35413,"score_linear":151655.0,"score_lmetric":333876},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":155910,"ongoing_decode_tokens":91379,"pending_prefill_tokens":5651,"num_requests":3,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":0,"new_prefill":35413,"score_linear":155910.0,"score_lmetric":123192},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":160162,"ongoing_decode_tokens":160162,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":35413,"score_linear":160162.0,"score_lmetric":106239}],"chosen_score_linear":21865.0,"chosen_score_lmetric":15590,"t_first_token":282013.25817962,"t_first_token_unix":1779712878.2721238,"t_done":282013.303850355,"t_done_unix":1779712878.3177936},{"request_id":"1370466:1:1370466:592","session_id":"1370466","input_length":7710,"t_proxy_recv":282012.665663051,"t_decision_unix":1779712877.6796048,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":7710,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":7710,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49483,"ongoing_decode_tokens":49483,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1060,"cache_hit":0,"new_prefill":7710,"score_linear":49483.0,"score_lmetric":7710},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":70968,"ongoing_decode_tokens":41467,"pending_prefill_tokens":7997,"num_requests":3,"active_p_offloads":0,"cached_blocks":1225,"cache_hit":0,"new_prefill":7710,"score_linear":70968.0,"score_lmetric":47121},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":7710,"score_linear":150402.0,"score_lmetric":260928},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":18301,"ongoing_decode_tokens":0,"pending_prefill_tokens":10621,"num_requests":1,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":7710,"score_linear":18301.0,"score_lmetric":18331},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":7710,"score_linear":199977.0,"score_lmetric":384981},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":7710,"score_linear":151655.0,"score_lmetric":250767},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":155910,"ongoing_decode_tokens":91379,"pending_prefill_tokens":5651,"num_requests":3,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":0,"new_prefill":7710,"score_linear":155910.0,"score_lmetric":40083},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":160162,"ongoing_decode_tokens":160162,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":7710,"score_linear":160162.0,"score_lmetric":23130}],"chosen_score_linear":49483.0,"chosen_score_lmetric":7710,"t_first_token":282013.212479528,"t_first_token_unix":1779712878.2264235,"t_done":282013.446385048,"t_done_unix":1779712878.4603283},{"request_id":"1368070:2:1370108:586","session_id":"1368070","input_length":29501,"t_proxy_recv":282011.324954929,"t_decision_unix":1779712876.3388958,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":21504,"affinity_cache_ratio":0.7289244432392122,"affinity_num_requests":2,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":21504,"estimated_new_tokens":7997,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93772,"ongoing_decode_tokens":93772,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":29501,"score_linear":93772.0,"score_lmetric":88503},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":41467,"ongoing_decode_tokens":18160,"pending_prefill_tokens":7947,"num_requests":2,"active_p_offloads":0,"cached_blocks":1225,"cache_hit":21504,"new_prefill":7997,"score_linear":19963.0,"score_lmetric":31888},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":29501,"score_linear":150402.0,"score_lmetric":304510},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":29501,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":29501,"score_linear":199977.0,"score_lmetric":450354},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":29501,"score_linear":151655.0,"score_lmetric":316140},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":0,"new_prefill":29501,"score_linear":80246.0,"score_lmetric":29501},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":177939,"ongoing_decode_tokens":177939,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":29501,"score_linear":177939.0,"score_lmetric":88503}],"chosen_score_linear":19963.0,"chosen_score_lmetric":31888,"t_first_token":282013.322738564,"t_first_token_unix":1779712878.336682,"t_done":282013.56222726,"t_done_unix":1779712878.5761707},{"request_id":"1370269:1:1370269:591","session_id":"1370269","input_length":2381,"t_proxy_recv":282011.874804276,"t_decision_unix":1779712876.8887458,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":4762,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":2381,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":84677,"ongoing_decode_tokens":84677,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":2381,"score_linear":84677.0,"score_lmetric":4762},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":70968,"ongoing_decode_tokens":41467,"pending_prefill_tokens":7997,"num_requests":3,"active_p_offloads":0,"cached_blocks":1225,"cache_hit":0,"new_prefill":2381,"score_linear":70968.0,"score_lmetric":31134},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":2381,"score_linear":150402.0,"score_lmetric":250270},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":18301,"ongoing_decode_tokens":0,"pending_prefill_tokens":10621,"num_requests":1,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":2381,"score_linear":18301.0,"score_lmetric":13002},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":2381,"score_linear":199977.0,"score_lmetric":368994},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":2381,"score_linear":151655.0,"score_lmetric":234780},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":155910,"ongoing_decode_tokens":80246,"pending_prefill_tokens":16784,"num_requests":3,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":0,"new_prefill":2381,"score_linear":155910.0,"score_lmetric":57495},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":157781,"ongoing_decode_tokens":157781,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":2381,"score_linear":157781.0,"score_lmetric":4762}],"chosen_score_linear":157781.0,"chosen_score_lmetric":4762,"t_first_token":282012.208753446,"t_first_token_unix":1779712877.2226973,"t_done":282013.589990729,"t_done_unix":1779712878.6039338},{"request_id":"1366128:3:1369902:585","session_id":"1366128","input_length":23307,"t_proxy_recv":282010.614240054,"t_decision_unix":1779712875.6281812,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":15360,"affinity_cache_ratio":0.6590294761230532,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":15360,"estimated_new_tokens":7947,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93772,"ongoing_decode_tokens":0,"pending_prefill_tokens":50252,"num_requests":3,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":23307,"score_linear":93772.0,"score_lmetric":220677},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":18160,"ongoing_decode_tokens":18160,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1225,"cache_hit":15360,"new_prefill":7947,"score_linear":2800.0,"score_lmetric":7947},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":23307,"score_linear":150402.0,"score_lmetric":292122},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":7712,"ongoing_decode_tokens":0,"pending_prefill_tokens":7712,"num_requests":1,"active_p_offloads":0,"cached_blocks":1521,"cache_hit":0,"new_prefill":23307,"score_linear":7712.0,"score_lmetric":31019},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":23307,"score_linear":199977.0,"score_lmetric":431772},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":23307,"score_linear":151655.0,"score_lmetric":297558},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":0,"new_prefill":23307,"score_linear":80246.0,"score_lmetric":23307},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":177939,"ongoing_decode_tokens":177939,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":23307,"score_linear":177939.0,"score_lmetric":69921}],"chosen_score_linear":2800.0,"chosen_score_lmetric":7947,"t_first_token":282011.862121283,"t_first_token_unix":1779712876.8760645,"t_done":282013.67390113,"t_done_unix":1779712878.6878443},{"request_id":"1363943:3:1369535:582","session_id":"1363943","input_length":18160,"t_proxy_recv":282009.321691706,"t_decision_unix":1779712874.335633,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":12800,"affinity_cache_ratio":0.7048458149779736,"affinity_num_requests":0,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":5360,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93772,"ongoing_decode_tokens":0,"pending_prefill_tokens":50252,"num_requests":3,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":8704,"new_prefill":9456,"score_linear":85068.0,"score_lmetric":179124},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1225,"cache_hit":12800,"new_prefill":5360,"score_linear":-12800.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":18160,"score_linear":150402.0,"score_lmetric":281828},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":24716,"ongoing_decode_tokens":24716,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1502,"cache_hit":4096,"new_prefill":14064,"score_linear":20620.0,"score_lmetric":28128},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":18160,"score_linear":199977.0,"score_lmetric":416331},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":18160,"score_linear":151655.0,"score_lmetric":282117},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":4096,"new_prefill":14064,"score_linear":76150.0,"score_lmetric":14064},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":198798,"ongoing_decode_tokens":198798,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":4096,"new_prefill":14064,"score_linear":194702.0,"score_lmetric":42192}],"chosen_score_linear":-12800.0,"chosen_score_lmetric":0,"t_first_token":282010.005396295,"t_first_token_unix":1779712875.0193398,"t_done":282014.183874849,"t_done_unix":1779712879.197818},{"request_id":"1353821:2:1357420:517","session_id":"1353821","input_length":27805,"t_proxy_recv":281966.543136902,"t_decision_unix":1779712831.5570781,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":27648,"affinity_cache_ratio":0.9943535335371336,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":27648,"estimated_new_tokens":157,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1036,"cache_hit":0,"new_prefill":27805,"score_linear":68259.0,"score_lmetric":27805},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":48193,"ongoing_decode_tokens":48193,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":982,"cache_hit":0,"new_prefill":27805,"score_linear":48193.0,"score_lmetric":27805},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":122597,"ongoing_decode_tokens":0,"pending_prefill_tokens":122597,"num_requests":1,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":27648,"new_prefill":157,"score_linear":94949.0,"score_lmetric":122754},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":34891,"ongoing_decode_tokens":21371,"pending_prefill_tokens":13520,"num_requests":2,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":0,"new_prefill":27805,"score_linear":34891.0,"score_lmetric":82650},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":27805,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":27805,"score_linear":83526.0,"score_lmetric":111331},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":27805,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":84285,"ongoing_decode_tokens":0,"pending_prefill_tokens":36157,"num_requests":2,"active_p_offloads":0,"cached_blocks":1217,"cache_hit":0,"new_prefill":27805,"score_linear":84285.0,"score_lmetric":127924}],"chosen_score_linear":94949.0,"chosen_score_lmetric":122754,"t_first_token":282012.827458021,"t_first_token_unix":1779712877.841401,"t_done":282014.421707261,"t_done_unix":1779712879.4356503},{"request_id":"1313181:3:1364311:547","session_id":"1313181","input_length":59514,"t_proxy_recv":281994.855261159,"t_decision_unix":1779712859.8692021,"policy":"unified","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":24064,"affinity_cache_ratio":0.4043418355344961,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":35450,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1055,"cache_hit":0,"new_prefill":59514,"score_linear":68259.0,"score_lmetric":59514},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1074,"cache_hit":0,"new_prefill":59514,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":59514,"score_linear":150402.0,"score_lmetric":364536},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":91690,"ongoing_decode_tokens":0,"pending_prefill_tokens":78378,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":59514,"score_linear":91690.0,"score_lmetric":275784},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":59514,"score_linear":199977.0,"score_lmetric":540393},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":59514,"score_linear":62685.0,"score_lmetric":122199},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":94143,"ongoing_decode_tokens":94143,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":59514,"score_linear":94143.0,"score_lmetric":119028},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":24064,"new_prefill":35450,"score_linear":-24064.0,"score_lmetric":0}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":282006.344709489,"t_first_token_unix":1779712871.3586528,"t_done":282014.530012959,"t_done_unix":1779712879.5439563},{"request_id":"1340278:2:1367225:564","session_id":"1340278","input_length":49483,"t_proxy_recv":282001.328857835,"t_decision_unix":1779712866.3427994,"policy":"unified","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":0,"affinity_cache_hit":48640,"affinity_cache_ratio":0.9829638461693915,"affinity_num_requests":4,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":49483,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":49483,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":34226,"ongoing_decode_tokens":0,"pending_prefill_tokens":23474,"num_requests":2,"active_p_offloads":0,"cached_blocks":1114,"cache_hit":0,"new_prefill":49483,"score_linear":34226.0,"score_lmetric":145914},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":49483,"score_linear":150402.0,"score_lmetric":344474},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":127991,"ongoing_decode_tokens":0,"pending_prefill_tokens":80887,"num_requests":3,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":49483,"score_linear":127991.0,"score_lmetric":391110},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":49483,"score_linear":199977.0,"score_lmetric":510300},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1535,"cache_hit":0,"new_prefill":49483,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":49483,"score_linear":80246.0,"score_lmetric":49483},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":218824,"ongoing_decode_tokens":0,"pending_prefill_tokens":43720,"num_requests":4,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":48640,"new_prefill":843,"score_linear":170184.0,"score_lmetric":178252}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282010.676743882,"t_first_token_unix":1779712875.6906872,"t_done":282014.671512416,"t_done_unix":1779712879.6854556},{"request_id":"1367424:2:1370211:588","session_id":"1367424","input_length":11133,"t_proxy_recv":282011.618125094,"t_decision_unix":1779712876.6320662,"policy":"unified","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":6,"affinity_cache_hit":3072,"affinity_cache_ratio":0.27593640528159524,"affinity_num_requests":3,"avg_num_requests":2.125,"fallback_score":11133,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":11133,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":84677,"ongoing_decode_tokens":84677,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":11133,"score_linear":84677.0,"score_lmetric":22266},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":70968,"ongoing_decode_tokens":18160,"pending_prefill_tokens":15944,"num_requests":3,"active_p_offloads":0,"cached_blocks":1225,"cache_hit":0,"new_prefill":11133,"score_linear":70968.0,"score_lmetric":81231},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":11133,"score_linear":150402.0,"score_lmetric":267774},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":18301,"ongoing_decode_tokens":0,"pending_prefill_tokens":10621,"num_requests":1,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":11133,"score_linear":18301.0,"score_lmetric":21754},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":11133,"score_linear":199977.0,"score_lmetric":395250},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":3072,"new_prefill":8061,"score_linear":148583.0,"score_lmetric":251820},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":0,"new_prefill":11133,"score_linear":80246.0,"score_lmetric":11133},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":157781,"ongoing_decode_tokens":157781,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":11133,"score_linear":157781.0,"score_lmetric":22266}],"chosen_score_linear":80246.0,"chosen_score_lmetric":11133,"t_first_token":282012.593677206,"t_first_token_unix":1779712877.607621,"t_done":282014.809856119,"t_done_unix":1779712879.8237996},{"request_id":"1340278:3:1368072:576","session_id":"1340278","input_length":50124,"t_proxy_recv":282014.702019587,"t_decision_unix":1779712879.7159607,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9908227595563004,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":460,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1172,"cache_hit":49664,"new_prefill":460,"score_linear":-49664.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":50124,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150530,"ongoing_decode_tokens":150530,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":50124,"score_linear":150530.0,"score_lmetric":100248},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":18301,"ongoing_decode_tokens":18301,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":50124,"score_linear":18301.0,"score_lmetric":50124},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":50124,"score_linear":199977.0,"score_lmetric":512223},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":50124,"score_linear":151655.0,"score_lmetric":378009},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":155910,"ongoing_decode_tokens":155910,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":0,"new_prefill":50124,"score_linear":155910.0,"score_lmetric":150372},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":140971,"ongoing_decode_tokens":140971,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":48640,"new_prefill":1484,"score_linear":92331.0,"score_lmetric":2968}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":282014.908546041,"t_first_token_unix":1779712879.9224894,"t_done":282014.948309329,"t_done_unix":1779712879.9622529},{"request_id":"1277428:6:1371101:593","session_id":"1277428","input_length":36583,"t_proxy_recv":282014.705767749,"t_decision_unix":1779712879.7197094,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":36352,"affinity_cache_ratio":0.9936855916682612,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":36352,"estimated_new_tokens":231,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":50124,"ongoing_decode_tokens":0,"pending_prefill_tokens":460,"num_requests":1,"active_p_offloads":0,"cached_blocks":1172,"cache_hit":0,"new_prefill":36583,"score_linear":50124.0,"score_lmetric":37043},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":4608,"new_prefill":31975,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150530,"ongoing_decode_tokens":150530,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":36583,"score_linear":150530.0,"score_lmetric":73166},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":18301,"ongoing_decode_tokens":18301,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":36352,"new_prefill":231,"score_linear":-18051.0,"score_lmetric":231},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":36583,"score_linear":199977.0,"score_lmetric":471600},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":36583,"score_linear":151655.0,"score_lmetric":337386},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":155910,"ongoing_decode_tokens":155910,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":0,"new_prefill":36583,"score_linear":155910.0,"score_lmetric":109749},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":140971,"ongoing_decode_tokens":140971,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":36583,"score_linear":140971.0,"score_lmetric":73166}],"chosen_score_linear":-18051.0,"chosen_score_lmetric":231,"t_first_token":282014.846406563,"t_first_token_unix":1779712879.8603501,"t_done":282015.272697261,"t_done_unix":1779712880.2866406},{"request_id":"1353821:3:1360898:529","session_id":"1353821","input_length":27933,"t_proxy_recv":282014.429056195,"t_decision_unix":1779712879.4429977,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":27648,"affinity_cache_ratio":0.98979701428418,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":27648,"estimated_new_tokens":285,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49483,"ongoing_decode_tokens":49483,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1075,"cache_hit":0,"new_prefill":27933,"score_linear":49483.0,"score_lmetric":27933},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":27933,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":122597,"ongoing_decode_tokens":122597,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":27648,"new_prefill":285,"score_linear":94949.0,"score_lmetric":285},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":18301,"ongoing_decode_tokens":18301,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":27933,"score_linear":18301.0,"score_lmetric":27933},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":27933,"score_linear":199977.0,"score_lmetric":445650},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":27933,"score_linear":151655.0,"score_lmetric":311436},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":155910,"ongoing_decode_tokens":91379,"pending_prefill_tokens":5651,"num_requests":3,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":0,"new_prefill":27933,"score_linear":155910.0,"score_lmetric":100752},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":157781,"ongoing_decode_tokens":157781,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1389,"cache_hit":0,"new_prefill":27933,"score_linear":157781.0,"score_lmetric":55866}],"chosen_score_linear":94949.0,"chosen_score_lmetric":285,"t_first_token":282014.49937745,"t_first_token_unix":1779712879.5133204,"t_done":282015.384827882,"t_done_unix":1779712880.3987713},{"request_id":"1356885:1:1356885:514","session_id":"1356885","input_length":122597,"t_proxy_recv":281964.904285129,"t_decision_unix":1779712829.9182265,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":122597,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1036,"cache_hit":0,"new_prefill":122597,"score_linear":68259.0,"score_lmetric":122597},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":48193,"ongoing_decode_tokens":48193,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":982,"cache_hit":0,"new_prefill":122597,"score_linear":48193.0,"score_lmetric":122597},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":122597,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1295,"cache_hit":6656,"new_prefill":115941,"score_linear":14715.0,"score_lmetric":115941},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1300,"cache_hit":0,"new_prefill":122597,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":122597,"score_linear":83526.0,"score_lmetric":206123},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78089,"ongoing_decode_tokens":78089,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":122597,"score_linear":78089.0,"score_lmetric":122597},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":84285,"ongoing_decode_tokens":0,"pending_prefill_tokens":36157,"num_requests":2,"active_p_offloads":0,"cached_blocks":1217,"cache_hit":0,"new_prefill":122597,"score_linear":84285.0,"score_lmetric":317508}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282012.826132877,"t_first_token_unix":1779712877.8400764,"t_done":282015.515348906,"t_done_unix":1779712880.5292923},{"request_id":"1353821:4:1363399:541","session_id":"1353821","input_length":28093,"t_proxy_recv":282015.393714998,"t_decision_unix":1779712880.4076567,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":27648,"affinity_cache_ratio":0.984159755099135,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":27648,"estimated_new_tokens":445,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1172,"cache_hit":0,"new_prefill":28093,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":28093,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":122597,"ongoing_decode_tokens":122597,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":27648,"new_prefill":445,"score_linear":94949.0,"score_lmetric":445},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":18301,"ongoing_decode_tokens":18301,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":28093,"score_linear":18301.0,"score_lmetric":28093},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":28093,"score_linear":199977.0,"score_lmetric":446130},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":28093,"score_linear":151655.0,"score_lmetric":311916},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":177591,"ongoing_decode_tokens":144777,"pending_prefill_tokens":6190,"num_requests":3,"active_p_offloads":0,"cached_blocks":1326,"cache_hit":0,"new_prefill":28093,"score_linear":177591.0,"score_lmetric":102849},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":163963,"ongoing_decode_tokens":140971,"pending_prefill_tokens":3024,"num_requests":3,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":28093,"score_linear":163963.0,"score_lmetric":93351}],"chosen_score_linear":94949.0,"chosen_score_lmetric":445,"t_first_token":282015.509166239,"t_first_token_unix":1779712880.5231094,"t_done":282015.663254224,"t_done_unix":1779712880.6771972},{"request_id":"1370129:1:1370129:587","session_id":"1370129","input_length":18301,"t_proxy_recv":282011.395747935,"t_decision_unix":1779712876.4096894,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":0,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":10621,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":84677,"ongoing_decode_tokens":84677,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":7680,"new_prefill":10621,"score_linear":76997.0,"score_lmetric":21242},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":70968,"ongoing_decode_tokens":18160,"pending_prefill_tokens":15944,"num_requests":3,"active_p_offloads":0,"cached_blocks":1225,"cache_hit":4096,"new_prefill":14205,"score_linear":66872.0,"score_lmetric":90447},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":18301,"score_linear":150402.0,"score_lmetric":282110},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":7680,"new_prefill":10621,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":18301,"score_linear":199977.0,"score_lmetric":416754},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":18301,"score_linear":151655.0,"score_lmetric":282540},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":7680,"new_prefill":10621,"score_linear":72566.0,"score_lmetric":10621},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":177939,"ongoing_decode_tokens":177939,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":7680,"new_prefill":10621,"score_linear":170259.0,"score_lmetric":31863}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":282012.686267385,"t_first_token_unix":1779712877.7002106,"t_done":282016.007782952,"t_done_unix":1779712881.0217264},{"request_id":"1313181:4:1318843:342","session_id":"1313181","input_length":42704,"t_proxy_recv":282014.545791636,"t_decision_unix":1779712879.559733,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":42496,"affinity_cache_ratio":0.9951292618958412,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":42496,"estimated_new_tokens":208,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49483,"ongoing_decode_tokens":49483,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1075,"cache_hit":0,"new_prefill":42704,"score_linear":49483.0,"score_lmetric":42704},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":42704,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150530,"ongoing_decode_tokens":150530,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":42704,"score_linear":150530.0,"score_lmetric":85408},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":18301,"ongoing_decode_tokens":18301,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":42704,"score_linear":18301.0,"score_lmetric":42704},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":42704,"score_linear":199977.0,"score_lmetric":489963},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":42704,"score_linear":151655.0,"score_lmetric":355749},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":155910,"ongoing_decode_tokens":91379,"pending_prefill_tokens":5651,"num_requests":3,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":0,"new_prefill":42704,"score_linear":155910.0,"score_lmetric":145065},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":42496,"new_prefill":208,"score_linear":55771.0,"score_lmetric":208}],"chosen_score_linear":55771.0,"chosen_score_lmetric":208,"t_first_token":282014.632343279,"t_first_token_unix":1779712879.6462862,"t_done":282016.514840563,"t_done_unix":1779712881.528784},{"request_id":"1371337:1:1371337:597","session_id":"1371337","input_length":9399,"t_proxy_recv":282015.535102571,"t_decision_unix":1779712880.5490441,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9399,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1172,"cache_hit":0,"new_prefill":9399,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":9399,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":28093,"ongoing_decode_tokens":28093,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1559,"cache_hit":0,"new_prefill":9399,"score_linear":28093.0,"score_lmetric":9399},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":18301,"ongoing_decode_tokens":18301,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":9399,"score_linear":18301.0,"score_lmetric":9399},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":9399,"score_linear":199977.0,"score_lmetric":390048},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":9399,"score_linear":151655.0,"score_lmetric":255834},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":177591,"ongoing_decode_tokens":144777,"pending_prefill_tokens":6190,"num_requests":3,"active_p_offloads":0,"cached_blocks":1326,"cache_hit":0,"new_prefill":9399,"score_linear":177591.0,"score_lmetric":46767},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":163963,"ongoing_decode_tokens":140971,"pending_prefill_tokens":3024,"num_requests":3,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":9399,"score_linear":163963.0,"score_lmetric":37269}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282016.244653665,"t_first_token_unix":1779712881.2585979,"t_done":282016.782951529,"t_done_unix":1779712881.7968948},{"request_id":"1371622:1:1371622:598","session_id":"1371622","input_length":13616,"t_proxy_recv":282016.540689145,"t_decision_unix":1779712881.5546308,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":13616,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":9399,"ongoing_decode_tokens":9399,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1172,"cache_hit":0,"new_prefill":13616,"score_linear":9399.0,"score_lmetric":13616},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":13616,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1559,"cache_hit":0,"new_prefill":13616,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":13616,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":11264,"new_prefill":2352,"score_linear":188713.0,"score_lmetric":368907},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":13616,"score_linear":151655.0,"score_lmetric":268485},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":177591,"ongoing_decode_tokens":177591,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1326,"cache_hit":0,"new_prefill":13616,"score_linear":177591.0,"score_lmetric":40848},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181263,"ongoing_decode_tokens":121259,"pending_prefill_tokens":17508,"num_requests":3,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":13616,"score_linear":181263.0,"score_lmetric":93372}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282017.701147694,"t_first_token_unix":1779712882.715091,"t_done":282018.448431158,"t_done_unix":1779712883.4623747},{"request_id":"1366128:4:1371629:599","session_id":"1366128","input_length":35037,"t_proxy_recv":282016.574082846,"t_decision_unix":1779712881.5880237,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":23040,"affinity_cache_ratio":0.6575905471358849,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":23040,"estimated_new_tokens":11997,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":9399,"ongoing_decode_tokens":9399,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1172,"cache_hit":0,"new_prefill":35037,"score_linear":9399.0,"score_lmetric":35037},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":23040,"new_prefill":11997,"score_linear":-23040.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":13616,"num_requests":1,"active_p_offloads":0,"cached_blocks":1559,"cache_hit":0,"new_prefill":35037,"score_linear":13616.0,"score_lmetric":48653},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":35037,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":35037,"score_linear":199977.0,"score_lmetric":466962},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":35037,"score_linear":151655.0,"score_lmetric":332748},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":177591,"ongoing_decode_tokens":177591,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1326,"cache_hit":0,"new_prefill":35037,"score_linear":177591.0,"score_lmetric":105111},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181263,"ongoing_decode_tokens":121259,"pending_prefill_tokens":17508,"num_requests":3,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":35037,"score_linear":181263.0,"score_lmetric":157635}],"chosen_score_linear":-23040.0,"chosen_score_lmetric":0,"t_first_token":282019.103440446,"t_first_token_unix":1779712884.1173842,"t_done":282019.435210624,"t_done_unix":1779712884.449154},{"request_id":"1363943:4:1372250:603","session_id":"1363943","input_length":18396,"t_proxy_recv":282018.882593325,"t_decision_unix":1779712883.8965347,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":17920,"affinity_cache_ratio":0.974124809741248,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":17920,"estimated_new_tokens":476,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":17407,"ongoing_decode_tokens":0,"pending_prefill_tokens":17407,"num_requests":1,"active_p_offloads":0,"cached_blocks":1190,"cache_hit":8704,"new_prefill":9692,"score_linear":8703.0,"score_lmetric":27099},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35037,"ongoing_decode_tokens":0,"pending_prefill_tokens":11997,"num_requests":1,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":17920,"new_prefill":476,"score_linear":17117.0,"score_lmetric":12473},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":18396,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":38252,"ongoing_decode_tokens":0,"pending_prefill_tokens":1900,"num_requests":1,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":4096,"new_prefill":14300,"score_linear":34156.0,"score_lmetric":16200},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":18396,"score_linear":199977.0,"score_lmetric":417039},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":18396,"score_linear":151655.0,"score_lmetric":282825},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":204296,"ongoing_decode_tokens":177591,"pending_prefill_tokens":12369,"num_requests":4,"active_p_offloads":0,"cached_blocks":1326,"cache_hit":4096,"new_prefill":14300,"score_linear":200200.0,"score_lmetric":106676},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181263,"ongoing_decode_tokens":121259,"pending_prefill_tokens":17508,"num_requests":3,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":4096,"new_prefill":14300,"score_linear":177167.0,"score_lmetric":95424}],"chosen_score_linear":17117.0,"chosen_score_lmetric":12473,"t_first_token":282019.137731086,"t_first_token_unix":1779712884.1516743,"t_done":282019.726071358,"t_done_unix":1779712884.7400146},{"request_id":"1371929:1:1371929:601","session_id":"1371929","input_length":17407,"t_proxy_recv":282017.644048058,"t_decision_unix":1779712882.6579897,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":17407,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1190,"cache_hit":0,"new_prefill":17407,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35037,"ongoing_decode_tokens":0,"pending_prefill_tokens":11997,"num_requests":1,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":17407,"score_linear":35037.0,"score_lmetric":29404},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":13616,"num_requests":1,"active_p_offloads":0,"cached_blocks":1559,"cache_hit":0,"new_prefill":17407,"score_linear":13616.0,"score_lmetric":31023},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":17407,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":17407,"score_linear":199977.0,"score_lmetric":414072},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":17407,"score_linear":151655.0,"score_lmetric":279858},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":204296,"ongoing_decode_tokens":177591,"pending_prefill_tokens":12369,"num_requests":4,"active_p_offloads":0,"cached_blocks":1326,"cache_hit":0,"new_prefill":17407,"score_linear":204296.0,"score_lmetric":119104},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181263,"ongoing_decode_tokens":121259,"pending_prefill_tokens":17508,"num_requests":3,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":17407,"score_linear":181263.0,"score_lmetric":104745}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282019.342006996,"t_first_token_unix":1779712884.3559508,"t_done":282019.81299216,"t_done_unix":1779712884.8269355},{"request_id":"1342634:2:1370263:590","session_id":"1342634","input_length":64531,"t_proxy_recv":282011.855860703,"t_decision_unix":1779712876.869802,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9124296849576172,"affinity_num_requests":2,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":5651,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":84677,"ongoing_decode_tokens":84677,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":64531,"score_linear":84677.0,"score_lmetric":129062},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":70968,"ongoing_decode_tokens":18160,"pending_prefill_tokens":15944,"num_requests":3,"active_p_offloads":0,"cached_blocks":1225,"cache_hit":0,"new_prefill":64531,"score_linear":70968.0,"score_lmetric":241425},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":64531,"score_linear":150402.0,"score_lmetric":374570},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":18301,"ongoing_decode_tokens":0,"pending_prefill_tokens":10621,"num_requests":1,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":64531,"score_linear":18301.0,"score_lmetric":75152},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":64531,"score_linear":199977.0,"score_lmetric":555444},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":64531,"score_linear":151655.0,"score_lmetric":421230},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":91379,"ongoing_decode_tokens":80246,"pending_prefill_tokens":11133,"num_requests":2,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":58880,"new_prefill":5651,"score_linear":32499.0,"score_lmetric":33568},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":157781,"ongoing_decode_tokens":157781,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":64531,"score_linear":157781.0,"score_lmetric":129062}],"chosen_score_linear":32499.0,"chosen_score_lmetric":33568,"t_first_token":282014.605021853,"t_first_token_unix":1779712879.6189651,"t_done":282020.027762145,"t_done_unix":1779712885.0417054},{"request_id":"1367329:2:1371226:595","session_id":"1367329","input_length":32814,"t_proxy_recv":282015.106207045,"t_decision_unix":1779712880.120148,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":26624,"affinity_cache_ratio":0.8113610044493205,"affinity_num_requests":2,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":26624,"estimated_new_tokens":6190,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1172,"cache_hit":0,"new_prefill":32814,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":32814,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150530,"ongoing_decode_tokens":150530,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":32814,"score_linear":150530.0,"score_lmetric":65628},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":54884,"ongoing_decode_tokens":54884,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":32814,"score_linear":54884.0,"score_lmetric":65628},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":32814,"score_linear":199977.0,"score_lmetric":460293},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":32814,"score_linear":151655.0,"score_lmetric":326079},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":144777,"ongoing_decode_tokens":144777,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1326,"cache_hit":26624,"new_prefill":6190,"score_linear":118153.0,"score_lmetric":12380},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":140971,"ongoing_decode_tokens":140971,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":32814,"score_linear":140971.0,"score_lmetric":65628}],"chosen_score_linear":118153.0,"chosen_score_lmetric":12380,"t_first_token":282016.447061257,"t_first_token_unix":1779712881.4610047,"t_done":282020.854864702,"t_done_unix":1779712885.868808},{"request_id":"1372791:1:1372791:604","session_id":"1372791","input_length":4811,"t_proxy_recv":282020.650739439,"t_decision_unix":1779712885.6646805,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4811,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1224,"cache_hit":0,"new_prefill":4811,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1289,"cache_hit":0,"new_prefill":4811,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":4811,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":38252,"ongoing_decode_tokens":38252,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":4811,"score_linear":38252.0,"score_lmetric":4811},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":4811,"score_linear":199977.0,"score_lmetric":376284},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":4811,"score_linear":151655.0,"score_lmetric":242070},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":139765,"ongoing_decode_tokens":139765,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1337,"cache_hit":0,"new_prefill":4811,"score_linear":139765.0,"score_lmetric":14433},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181263,"ongoing_decode_tokens":121259,"pending_prefill_tokens":17508,"num_requests":3,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":4811,"score_linear":181263.0,"score_lmetric":66957}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282020.955386619,"t_first_token_unix":1779712885.96933,"t_done":282021.136375713,"t_done_unix":1779712886.1503186},{"request_id":"1362265:3:1373357:605","session_id":"1362265","input_length":10284,"t_proxy_recv":282022.651550151,"t_decision_unix":1779712887.6654918,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":9728,"affinity_cache_ratio":0.9459354336833917,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":9728,"estimated_new_tokens":556,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":6656,"new_prefill":3628,"score_linear":-6656.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1289,"cache_hit":9728,"new_prefill":556,"score_linear":-9728.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":10284,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":38252,"ongoing_decode_tokens":38252,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":6656,"new_prefill":3628,"score_linear":31596.0,"score_lmetric":3628},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":10284,"score_linear":199977.0,"score_lmetric":392703},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":151655,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":10284,"score_linear":151655.0,"score_lmetric":30852},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1349,"cache_hit":0,"new_prefill":10284,"score_linear":106951.0,"score_lmetric":20568},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181263,"ongoing_decode_tokens":181263,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":10284,"score_linear":181263.0,"score_lmetric":30852}],"chosen_score_linear":-9728.0,"chosen_score_lmetric":0,"t_first_token":282022.7038472,"t_first_token_unix":1779712887.717791,"t_done":282022.805829179,"t_done_unix":1779712887.8197725},{"request_id":"1277428:7:1372183:602","session_id":"1277428","input_length":38252,"t_proxy_recv":282018.61203743,"t_decision_unix":1779712883.6259787,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":36352,"affinity_cache_ratio":0.9503293945414619,"affinity_num_requests":0,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":36352,"estimated_new_tokens":1900,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":17407,"ongoing_decode_tokens":0,"pending_prefill_tokens":17407,"num_requests":1,"active_p_offloads":0,"cached_blocks":1190,"cache_hit":0,"new_prefill":38252,"score_linear":17407.0,"score_lmetric":55659},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35037,"ongoing_decode_tokens":0,"pending_prefill_tokens":11997,"num_requests":1,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":4608,"new_prefill":33644,"score_linear":30429.0,"score_lmetric":45641},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":38252,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":36352,"new_prefill":1900,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":38252,"score_linear":199977.0,"score_lmetric":476607},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":38252,"score_linear":151655.0,"score_lmetric":342393},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":204296,"ongoing_decode_tokens":177591,"pending_prefill_tokens":12369,"num_requests":4,"active_p_offloads":0,"cached_blocks":1326,"cache_hit":0,"new_prefill":38252,"score_linear":204296.0,"score_lmetric":202484},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181263,"ongoing_decode_tokens":121259,"pending_prefill_tokens":17508,"num_requests":3,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":38252,"score_linear":181263.0,"score_lmetric":167280}],"chosen_score_linear":-36352.0,"chosen_score_lmetric":0,"t_first_token":282019.133996163,"t_first_token_unix":1779712884.1479394,"t_done":282022.836639315,"t_done_unix":1779712887.8505826},{"request_id":"1367312:2:1368064:574","session_id":"1367312","input_length":15501,"t_proxy_recv":282004.270813604,"t_decision_unix":1779712869.2847548,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":13312,"affinity_cache_ratio":0.8587833043029482,"affinity_num_requests":1,"avg_num_requests":2.5,"fallback_score":null,"tie_break_used":false,"cache_hit":13312,"estimated_new_tokens":2189,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93772,"ongoing_decode_tokens":0,"pending_prefill_tokens":50252,"num_requests":3,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":15501,"score_linear":93772.0,"score_lmetric":197259},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":34226,"ongoing_decode_tokens":34226,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1116,"cache_hit":0,"new_prefill":15501,"score_linear":34226.0,"score_lmetric":31002},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":15501,"score_linear":150402.0,"score_lmetric":276510},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":127991,"ongoing_decode_tokens":91690,"pending_prefill_tokens":2509,"num_requests":3,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":15501,"score_linear":127991.0,"score_lmetric":54030},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":15501,"score_linear":199977.0,"score_lmetric":408354},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":13312,"new_prefill":2189,"score_linear":59607.0,"score_lmetric":75108},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":107083,"ongoing_decode_tokens":80246,"pending_prefill_tokens":26837,"num_requests":2,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":15501,"score_linear":107083.0,"score_lmetric":84676},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":218824,"ongoing_decode_tokens":0,"pending_prefill_tokens":43720,"num_requests":4,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":15501,"score_linear":218824.0,"score_lmetric":236884}],"chosen_score_linear":59607.0,"chosen_score_lmetric":75108,"t_first_token":282022.56415457,"t_first_token_unix":1779712887.5780978,"t_done":282023.343825909,"t_done_unix":1779712888.3577695},{"request_id":"1367957:1:1367957:572","session_id":"1367957","input_length":72919,"t_proxy_recv":282003.859498085,"t_decision_unix":1779712868.873433,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":72919,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93772,"ongoing_decode_tokens":0,"pending_prefill_tokens":50252,"num_requests":3,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":72919,"score_linear":93772.0,"score_lmetric":369513},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":44360,"ongoing_decode_tokens":44360,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1114,"cache_hit":0,"new_prefill":72919,"score_linear":44360.0,"score_lmetric":218757},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":18944,"new_prefill":53975,"score_linear":131458.0,"score_lmetric":353458},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":127991,"ongoing_decode_tokens":91690,"pending_prefill_tokens":2509,"num_requests":3,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":72919,"score_linear":127991.0,"score_lmetric":226284},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":72919,"score_linear":199977.0,"score_lmetric":580608},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":72919,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":107083,"ongoing_decode_tokens":80246,"pending_prefill_tokens":26837,"num_requests":2,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":72919,"score_linear":107083.0,"score_lmetric":199512},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":218824,"ongoing_decode_tokens":0,"pending_prefill_tokens":43720,"num_requests":4,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":72919,"score_linear":218824.0,"score_lmetric":466556}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282022.171831866,"t_first_token_unix":1779712887.1857772,"t_done":282023.64989053,"t_done_unix":1779712888.6638339},{"request_id":"1363093:2:1368700:580","session_id":"1363093","input_length":63235,"t_proxy_recv":282006.452461584,"t_decision_unix":1779712871.4664028,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":62464,"affinity_cache_ratio":0.9878073851506286,"affinity_num_requests":2,"avg_num_requests":2.75,"fallback_score":null,"tie_break_used":false,"cache_hit":62464,"estimated_new_tokens":771,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93772,"ongoing_decode_tokens":0,"pending_prefill_tokens":50252,"num_requests":3,"active_p_offloads":0,"cached_blocks":1059,"cache_hit":0,"new_prefill":63235,"score_linear":93772.0,"score_lmetric":340461},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":47636,"ongoing_decode_tokens":18815,"pending_prefill_tokens":26773,"num_requests":3,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":63235,"score_linear":47636.0,"score_lmetric":270024},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":63235,"score_linear":150402.0,"score_lmetric":371978},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":136836,"ongoing_decode_tokens":127991,"pending_prefill_tokens":8845,"num_requests":4,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":63235,"score_linear":136836.0,"score_lmetric":288320},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":63235,"score_linear":199977.0,"score_lmetric":551556},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":88420,"ongoing_decode_tokens":0,"pending_prefill_tokens":75108,"num_requests":2,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":62464,"new_prefill":771,"score_linear":25956.0,"score_lmetric":151758},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1305,"cache_hit":0,"new_prefill":63235,"score_linear":80246.0,"score_lmetric":63235},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":218824,"ongoing_decode_tokens":157781,"pending_prefill_tokens":7795,"num_requests":4,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":63235,"score_linear":218824.0,"score_lmetric":284120}],"chosen_score_linear":25956.0,"chosen_score_lmetric":151758,"t_first_token":282022.566477491,"t_first_token_unix":1779712887.5804205,"t_done":282024.339258465,"t_done_unix":1779712889.3532043},{"request_id":"1355484:7:1371284:596","session_id":"1355484","input_length":22992,"t_proxy_recv":282015.330650086,"t_decision_unix":1779712880.3445914,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":19968,"affinity_cache_ratio":0.8684759916492694,"affinity_num_requests":2,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":3024,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1172,"cache_hit":0,"new_prefill":22992,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":22992,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150530,"ongoing_decode_tokens":150530,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":22992,"score_linear":150530.0,"score_lmetric":45984},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":18301,"ongoing_decode_tokens":18301,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":22992,"score_linear":18301.0,"score_lmetric":22992},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":22992,"score_linear":199977.0,"score_lmetric":430827},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":22992,"score_linear":151655.0,"score_lmetric":296613},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":177591,"ongoing_decode_tokens":144777,"pending_prefill_tokens":6190,"num_requests":3,"active_p_offloads":0,"cached_blocks":1326,"cache_hit":0,"new_prefill":22992,"score_linear":177591.0,"score_lmetric":87546},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":140971,"ongoing_decode_tokens":140971,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":19968,"new_prefill":3024,"score_linear":121003.0,"score_lmetric":6048}],"chosen_score_linear":121003.0,"chosen_score_lmetric":6048,"t_first_token":282015.880899772,"t_first_token_unix":1779712880.894843,"t_done":282026.382865956,"t_done_unix":1779712891.396809},{"request_id":"1366128:5:1373829:610","session_id":"1366128","input_length":46764,"t_proxy_recv":282024.373012795,"t_decision_unix":1779712889.3869543,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":34816,"affinity_cache_ratio":0.7445043195620563,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":11948,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":46764,"score_linear":126340.0,"score_lmetric":173104},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":18676,"ongoing_decode_tokens":18676,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1290,"cache_hit":34816,"new_prefill":11948,"score_linear":-16140.0,"score_lmetric":11948},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":46764,"score_linear":79495.0,"score_lmetric":126259},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":0,"pending_prefill_tokens":37854,"num_requests":1,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":46764,"score_linear":56286.0,"score_lmetric":84618},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":46764,"score_linear":199977.0,"score_lmetric":502143},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":46764,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1349,"cache_hit":0,"new_prefill":46764,"score_linear":106951.0,"score_lmetric":93528},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181263,"ongoing_decode_tokens":181263,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":46764,"score_linear":181263.0,"score_lmetric":140292}],"chosen_score_linear":-16140.0,"chosen_score_lmetric":11948,"t_first_token":282027.698209776,"t_first_token_unix":1779712892.7121544,"t_done":282027.964993149,"t_done_unix":1779712892.9789376},{"request_id":"1363943:5:1373637:609","session_id":"1363943","input_length":18676,"t_proxy_recv":282023.717561716,"t_decision_unix":1779712888.731503,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":18432,"affinity_cache_ratio":0.9869351038766331,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":18432,"estimated_new_tokens":244,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":8704,"new_prefill":9972,"score_linear":117636.0,"score_lmetric":136312},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1290,"cache_hit":18432,"new_prefill":244,"score_linear":-18432.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":18676,"score_linear":79495.0,"score_lmetric":98171},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":0,"pending_prefill_tokens":37854,"num_requests":1,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":4096,"new_prefill":14580,"score_linear":52190.0,"score_lmetric":52434},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":18676,"score_linear":199977.0,"score_lmetric":417879},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":63235,"ongoing_decode_tokens":63235,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1713,"cache_hit":0,"new_prefill":18676,"score_linear":63235.0,"score_lmetric":18676},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1349,"cache_hit":4096,"new_prefill":14580,"score_linear":102855.0,"score_lmetric":29160},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181263,"ongoing_decode_tokens":181263,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":4096,"new_prefill":14580,"score_linear":177167.0,"score_lmetric":43740}],"chosen_score_linear":-18432.0,"chosen_score_lmetric":0,"t_first_token":282023.809525537,"t_first_token_unix":1779712888.8234696,"t_done":282028.243920331,"t_done_unix":1779712893.2578642},{"request_id":"1374465:1:1374465:613","session_id":"1374465","input_length":18497,"t_proxy_recv":282026.603118062,"t_decision_unix":1779712891.617059,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":36994,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":18497,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":18497,"score_linear":126340.0,"score_lmetric":144837},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":65440,"ongoing_decode_tokens":18676,"pending_prefill_tokens":11948,"num_requests":2,"active_p_offloads":0,"cached_blocks":1290,"cache_hit":0,"new_prefill":18497,"score_linear":65440.0,"score_lmetric":60890},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":18497,"score_linear":79495.0,"score_lmetric":97992},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":0,"pending_prefill_tokens":37854,"num_requests":1,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":18497,"score_linear":56286.0,"score_lmetric":56351},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":117888,"pending_prefill_tokens":2729,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":18497,"score_linear":199977.0,"score_lmetric":63678},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83398,"ongoing_decode_tokens":0,"pending_prefill_tokens":20422,"num_requests":2,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":18497,"score_linear":83398.0,"score_lmetric":77838},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1349,"cache_hit":0,"new_prefill":18497,"score_linear":106951.0,"score_lmetric":36994},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":158271,"ongoing_decode_tokens":158271,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1399,"cache_hit":0,"new_prefill":18497,"score_linear":158271.0,"score_lmetric":36994}],"chosen_score_linear":106951.0,"chosen_score_lmetric":36994,"t_first_token":282028.48923833,"t_first_token_unix":1779712893.5031824,"t_done":282028.989635901,"t_done_unix":1779712894.0035791},{"request_id":"1342921:2:1361022:530","session_id":"1342921","input_length":66032,"t_proxy_recv":281979.426552106,"t_decision_unix":1779712844.4404929,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":65536,"affinity_cache_ratio":0.992488490428883,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":65536,"estimated_new_tokens":496,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":0,"new_prefill":66032,"score_linear":68259.0,"score_lmetric":66032},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":989,"cache_hit":0,"new_prefill":66032,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":66032,"score_linear":150402.0,"score_lmetric":377572},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":66032,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":65536,"new_prefill":496,"score_linear":52352.0,"score_lmetric":118384},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":83526,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":66032,"score_linear":83526.0,"score_lmetric":66032},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":66032,"score_linear":80246.0,"score_lmetric":66032},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":111729,"ongoing_decode_tokens":0,"pending_prefill_tokens":40049,"num_requests":3,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":0,"new_prefill":66032,"score_linear":111729.0,"score_lmetric":318243}],"chosen_score_linear":52352.0,"chosen_score_lmetric":118384,"t_first_token":282027.784699217,"t_first_token_unix":1779712892.7986443,"t_done":282029.99340168,"t_done_unix":1779712895.0073454},{"request_id":"1364090:2:1371798:600","session_id":"1364090","input_length":26705,"t_proxy_recv":282017.179250292,"t_decision_unix":1779712882.1931918,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":14336,"affinity_cache_ratio":0.5368283093053735,"affinity_num_requests":3,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":12369,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1190,"cache_hit":7680,"new_prefill":19025,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35037,"ongoing_decode_tokens":0,"pending_prefill_tokens":11997,"num_requests":1,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":4096,"new_prefill":22609,"score_linear":30941.0,"score_lmetric":34606},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":13616,"num_requests":1,"active_p_offloads":0,"cached_blocks":1559,"cache_hit":0,"new_prefill":26705,"score_linear":13616.0,"score_lmetric":40321},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":7680,"new_prefill":19025,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":26705,"score_linear":199977.0,"score_lmetric":441966},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":26705,"score_linear":151655.0,"score_lmetric":307752},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":177591,"ongoing_decode_tokens":177591,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1326,"cache_hit":14336,"new_prefill":12369,"score_linear":163255.0,"score_lmetric":37107},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181263,"ongoing_decode_tokens":121259,"pending_prefill_tokens":17508,"num_requests":3,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":7680,"new_prefill":19025,"score_linear":173583.0,"score_lmetric":109599}],"chosen_score_linear":163255.0,"chosen_score_lmetric":37107,"t_first_token":282019.267841979,"t_first_token_unix":1779712884.281785,"t_done":282033.582162519,"t_done_unix":1779712898.5961058},{"request_id":"1374465:2:1376358:620","session_id":"1374465","input_length":21588,"t_proxy_recv":282033.005635783,"t_decision_unix":1779712898.019577,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":18432,"affinity_cache_ratio":0.8538076709282935,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":18432,"estimated_new_tokens":3156,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":21588,"score_linear":126340.0,"score_lmetric":147928},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77627,"ongoing_decode_tokens":58511,"pending_prefill_tokens":684,"num_requests":2,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":21588,"score_linear":77627.0,"score_lmetric":44544},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":21588,"score_linear":79495.0,"score_lmetric":101083},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":56286,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":21588,"score_linear":56286.0,"score_lmetric":21588},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":133945,"ongoing_decode_tokens":133945,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1314,"cache_hit":0,"new_prefill":21588,"score_linear":133945.0,"score_lmetric":43176},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160794,"ongoing_decode_tokens":83398,"pending_prefill_tokens":4692,"num_requests":4,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":21588,"score_linear":160794.0,"score_lmetric":105120},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":18432,"new_prefill":3156,"score_linear":88519.0,"score_lmetric":6312},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":196312,"ongoing_decode_tokens":158271,"pending_prefill_tokens":1177,"num_requests":3,"active_p_offloads":0,"cached_blocks":1399,"cache_hit":0,"new_prefill":21588,"score_linear":196312.0,"score_lmetric":68295}],"chosen_score_linear":88519.0,"chosen_score_lmetric":6312,"t_first_token":282033.578634806,"t_first_token_unix":1779712898.5925794,"t_done":282034.194864699,"t_done_unix":1779712899.2088082},{"request_id":"1363943:6:1375416:618","session_id":"1363943","input_length":19116,"t_proxy_recv":282029.911629897,"t_decision_unix":1779712894.925571,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":18432,"affinity_cache_ratio":0.9642184557438794,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":18432,"estimated_new_tokens":684,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":8704,"new_prefill":10412,"score_linear":117636.0,"score_lmetric":136752},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":58511,"ongoing_decode_tokens":0,"pending_prefill_tokens":11919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":18432,"new_prefill":684,"score_linear":40079.0,"score_lmetric":12603},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":19116,"score_linear":79495.0,"score_lmetric":98611},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":0,"pending_prefill_tokens":37854,"num_requests":1,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":4096,"new_prefill":15020,"score_linear":52190.0,"score_lmetric":52874},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":199977,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":19116,"score_linear":199977.0,"score_lmetric":57348},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160794,"ongoing_decode_tokens":83398,"pending_prefill_tokens":4692,"num_requests":4,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":19116,"score_linear":160794.0,"score_lmetric":95232},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":4096,"new_prefill":15020,"score_linear":102855.0,"score_lmetric":30040},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":158271,"ongoing_decode_tokens":158271,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1399,"cache_hit":4096,"new_prefill":15020,"score_linear":154175.0,"score_lmetric":30040}],"chosen_score_linear":40079.0,"chosen_score_lmetric":12603,"t_first_token":282033.06615601,"t_first_token_unix":1779712898.0800993,"t_done":282034.60687029,"t_done_unix":1779712899.6208138},{"request_id":"1355951:3:1375866:619","session_id":"1355951","input_length":38041,"t_proxy_recv":282031.550068599,"t_decision_unix":1779712896.56401,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":36864,"affinity_cache_ratio":0.9690596987460898,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":36864,"estimated_new_tokens":1177,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":38041,"score_linear":126340.0,"score_lmetric":164381},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77627,"ongoing_decode_tokens":0,"pending_prefill_tokens":12603,"num_requests":2,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":38041,"score_linear":77627.0,"score_lmetric":101288},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":38041,"score_linear":79495.0,"score_lmetric":117536},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":0,"pending_prefill_tokens":37854,"num_requests":1,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":38041,"score_linear":56286.0,"score_lmetric":75895},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":133945,"ongoing_decode_tokens":133945,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1314,"cache_hit":0,"new_prefill":38041,"score_linear":133945.0,"score_lmetric":76082},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160794,"ongoing_decode_tokens":83398,"pending_prefill_tokens":4692,"num_requests":4,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":38041,"score_linear":160794.0,"score_lmetric":170932},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":38041,"score_linear":106951.0,"score_lmetric":76082},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":158271,"ongoing_decode_tokens":158271,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1399,"cache_hit":36864,"new_prefill":1177,"score_linear":121407.0,"score_lmetric":2354}],"chosen_score_linear":121407.0,"chosen_score_lmetric":2354,"t_first_token":282034.062137901,"t_first_token_unix":1779712899.0760818,"t_done":282035.000970503,"t_done_unix":1779712900.0149138},{"request_id":"1366128:6:1375094:616","session_id":"1366128","input_length":58511,"t_proxy_recv":282028.877465355,"t_decision_unix":1779712893.8914065,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":46592,"affinity_cache_ratio":0.7962947138144965,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":46592,"estimated_new_tokens":11919,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":58511,"score_linear":126340.0,"score_lmetric":184851},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":46592,"new_prefill":11919,"score_linear":-46592.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":58511,"score_linear":79495.0,"score_lmetric":138006},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":0,"pending_prefill_tokens":37854,"num_requests":1,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":58511,"score_linear":56286.0,"score_lmetric":96365},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":183920,"pending_prefill_tokens":2233,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":58511,"score_linear":199977.0,"score_lmetric":182232},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124906,"ongoing_decode_tokens":83398,"pending_prefill_tokens":4644,"num_requests":3,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":58511,"score_linear":124906.0,"score_lmetric":189465},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125448,"ongoing_decode_tokens":125448,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1349,"cache_hit":0,"new_prefill":58511,"score_linear":125448.0,"score_lmetric":175533},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":158271,"ongoing_decode_tokens":158271,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1399,"cache_hit":0,"new_prefill":58511,"score_linear":158271.0,"score_lmetric":117022}],"chosen_score_linear":-46592.0,"chosen_score_lmetric":0,"t_first_token":282033.003300372,"t_first_token_unix":1779712898.0172446,"t_done":282035.029907171,"t_done_unix":1779712900.0438502},{"request_id":"1371337:2:1376534:621","session_id":"1371337","input_length":24323,"t_proxy_recv":282033.670417695,"t_decision_unix":1779712898.684359,"policy":"unified","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":3,"affinity_cache_hit":9216,"affinity_cache_ratio":0.3789006290342474,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":24323,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":24323,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":9216,"new_prefill":15107,"score_linear":117124.0,"score_lmetric":141447},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77627,"ongoing_decode_tokens":77627,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":24323,"score_linear":77627.0,"score_lmetric":48646},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":24323,"score_linear":79495.0,"score_lmetric":103818},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":56286,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":24323,"score_linear":56286.0,"score_lmetric":24323},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":133945,"ongoing_decode_tokens":133945,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1314,"cache_hit":0,"new_prefill":24323,"score_linear":133945.0,"score_lmetric":48646},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160794,"ongoing_decode_tokens":83398,"pending_prefill_tokens":4692,"num_requests":4,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":24323,"score_linear":160794.0,"score_lmetric":116060},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":101834,"ongoing_decode_tokens":101834,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1411,"cache_hit":0,"new_prefill":24323,"score_linear":101834.0,"score_lmetric":48646},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":196312,"ongoing_decode_tokens":158271,"pending_prefill_tokens":1177,"num_requests":3,"active_p_offloads":0,"cached_blocks":1399,"cache_hit":0,"new_prefill":24323,"score_linear":196312.0,"score_lmetric":76500}],"chosen_score_linear":56286.0,"chosen_score_lmetric":24323,"t_first_token":282036.460949301,"t_first_token_unix":1779712901.4748924,"t_done":282037.429322036,"t_done_unix":1779712902.4432652},{"request_id":"1360649:1:1360649:527","session_id":"1360649","input_length":117888,"t_proxy_recv":281978.173161578,"t_decision_unix":1779712843.1871028,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":117888,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":0,"new_prefill":117888,"score_linear":68259.0,"score_lmetric":117888},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":989,"cache_hit":0,"new_prefill":117888,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":117888,"score_linear":150402.0,"score_lmetric":481284},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":117888,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":117888,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":117888,"score_linear":83526.0,"score_lmetric":201414},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":117888,"score_linear":80246.0,"score_lmetric":117888},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":78251,"ongoing_decode_tokens":0,"pending_prefill_tokens":34731,"num_requests":2,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":0,"new_prefill":117888,"score_linear":78251.0,"score_lmetric":305238}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282024.673793902,"t_first_token_unix":1779712889.6877372,"t_done":282037.514415769,"t_done_unix":1779712902.5283592},{"request_id":"1377061:1:1377061:622","session_id":"1377061","input_length":16241,"t_proxy_recv":282035.632312874,"t_decision_unix":1779712900.6462543,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":16241,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":16241,"score_linear":126340.0,"score_lmetric":142581},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1337,"cache_hit":0,"new_prefill":16241,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":16241,"score_linear":79495.0,"score_lmetric":95736},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":80609,"ongoing_decode_tokens":56286,"pending_prefill_tokens":24323,"num_requests":2,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":16241,"score_linear":80609.0,"score_lmetric":81128},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":133945,"ongoing_decode_tokens":133945,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1314,"cache_hit":0,"new_prefill":16241,"score_linear":133945.0,"score_lmetric":32482},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160794,"ongoing_decode_tokens":124906,"pending_prefill_tokens":48,"num_requests":4,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":16241,"score_linear":160794.0,"score_lmetric":65156},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":16241,"score_linear":80246.0,"score_lmetric":16241},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":158271,"ongoing_decode_tokens":158271,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1401,"cache_hit":0,"new_prefill":16241,"score_linear":158271.0,"score_lmetric":32482}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282037.12228031,"t_first_token_unix":1779712902.1362243,"t_done":282037.559783602,"t_done_unix":1779712902.5737267},{"request_id":"1313181:4:1368020:573","session_id":"1313181","input_length":60004,"t_proxy_recv":282016.530618937,"t_decision_unix":1779712881.5445602,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":42496,"affinity_cache_ratio":0.7082194520365309,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":42496,"estimated_new_tokens":17508,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":9399,"ongoing_decode_tokens":9399,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1172,"cache_hit":0,"new_prefill":60004,"score_linear":9399.0,"score_lmetric":60004},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":60004,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1559,"cache_hit":0,"new_prefill":60004,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":60004,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":60004,"score_linear":199977.0,"score_lmetric":541863},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":0,"pending_prefill_tokens":75879,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":60004,"score_linear":151655.0,"score_lmetric":407649},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":177591,"ongoing_decode_tokens":177591,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1326,"cache_hit":0,"new_prefill":60004,"score_linear":177591.0,"score_lmetric":180012},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121259,"ongoing_decode_tokens":121259,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":42496,"new_prefill":17508,"score_linear":78763.0,"score_lmetric":35016}],"chosen_score_linear":78763.0,"chosen_score_lmetric":35016,"t_first_token":282022.397907943,"t_first_token_unix":1779712887.4118514,"t_done":282037.594852176,"t_done_unix":1779712902.6087954},{"request_id":"1363943:7:1377305:623","session_id":"1363943","input_length":20727,"t_proxy_recv":282036.557799453,"t_decision_unix":1779712901.5717409,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":18944,"affinity_cache_ratio":0.9139769382930477,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":18944,"estimated_new_tokens":1783,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":8704,"new_prefill":12023,"score_linear":117636.0,"score_lmetric":138363},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":16241,"ongoing_decode_tokens":0,"pending_prefill_tokens":16241,"num_requests":1,"active_p_offloads":0,"cached_blocks":1337,"cache_hit":18944,"new_prefill":1783,"score_linear":-2703.0,"score_lmetric":18024},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":20727,"score_linear":79495.0,"score_lmetric":100222},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":80609,"ongoing_decode_tokens":80609,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":4096,"new_prefill":16631,"score_linear":76513.0,"score_lmetric":33262},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":133945,"ongoing_decode_tokens":133945,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1314,"cache_hit":0,"new_prefill":20727,"score_linear":133945.0,"score_lmetric":41454},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160794,"ongoing_decode_tokens":124906,"pending_prefill_tokens":48,"num_requests":4,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":20727,"score_linear":160794.0,"score_lmetric":83100},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":4096,"new_prefill":16631,"score_linear":76150.0,"score_lmetric":16631},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":158271,"ongoing_decode_tokens":158271,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1401,"cache_hit":4096,"new_prefill":16631,"score_linear":154175.0,"score_lmetric":33262}],"chosen_score_linear":-2703.0,"chosen_score_lmetric":18024,"t_first_token":282037.386930593,"t_first_token_unix":1779712902.4008741,"t_done":282038.573023533,"t_done_unix":1779712903.5869668},{"request_id":"1305906:2:1363233:539","session_id":"1305906","input_length":16057,"t_proxy_recv":281987.222580532,"t_decision_unix":1779712852.2365217,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":13824,"affinity_cache_ratio":0.8609329264495236,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":13824,"estimated_new_tokens":2233,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":0,"new_prefill":16057,"score_linear":68259.0,"score_lmetric":16057},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":0,"new_prefill":16057,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":16057,"score_linear":150402.0,"score_lmetric":277622},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":16057,"score_linear":76969.0,"score_lmetric":93026},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":183920,"ongoing_decode_tokens":0,"pending_prefill_tokens":118384,"num_requests":2,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":13824,"new_prefill":2233,"score_linear":170096.0,"score_lmetric":241234},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":16057,"score_linear":62685.0,"score_lmetric":78742},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":16057,"score_linear":80246.0,"score_lmetric":16057},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":78251,"ongoing_decode_tokens":78251,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1297,"cache_hit":0,"new_prefill":16057,"score_linear":78251.0,"score_lmetric":32114}],"chosen_score_linear":170096.0,"chosen_score_lmetric":241234,"t_first_token":282029.194027677,"t_first_token_unix":1779712894.207971,"t_done":282038.574217584,"t_done_unix":1779712903.5881624},{"request_id":"1360649:2:1367022:561","session_id":"1360649","input_length":119052,"t_proxy_recv":282037.540848637,"t_decision_unix":1779712902.5547898,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":118272,"affinity_cache_ratio":0.9934482411047273,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":118272,"estimated_new_tokens":780,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":119052,"score_linear":126340.0,"score_lmetric":245392},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":36968,"ongoing_decode_tokens":36968,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1337,"cache_hit":0,"new_prefill":119052,"score_linear":36968.0,"score_lmetric":238104},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":119052,"score_linear":79495.0,"score_lmetric":198547},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":56286,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":119052,"score_linear":56286.0,"score_lmetric":119052},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":16057,"ongoing_decode_tokens":16057,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1545,"cache_hit":118272,"new_prefill":780,"score_linear":-102215.0,"score_lmetric":780},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160794,"ongoing_decode_tokens":124906,"pending_prefill_tokens":48,"num_requests":4,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":4096,"new_prefill":114956,"score_linear":156698.0,"score_lmetric":460016},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":119052,"score_linear":80246.0,"score_lmetric":119052},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":158271,"ongoing_decode_tokens":158271,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1401,"cache_hit":0,"new_prefill":119052,"score_linear":158271.0,"score_lmetric":238104}],"chosen_score_linear":-102215.0,"chosen_score_lmetric":780,"t_first_token":282038.146345647,"t_first_token_unix":1779712903.1602893,"t_done":282039.519707122,"t_done_unix":1779712904.5336506},{"request_id":"1372791:2:1377698:624","session_id":"1372791","input_length":14851,"t_proxy_recv":282037.871183118,"t_decision_unix":1779712902.8851247,"policy":"unified","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":3,"affinity_cache_hit":4608,"affinity_cache_ratio":0.3102821358831055,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":14851,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":14851,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":4608,"new_prefill":10243,"score_linear":121732.0,"score_lmetric":136583},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":20727,"ongoing_decode_tokens":20727,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1368,"cache_hit":0,"new_prefill":14851,"score_linear":20727.0,"score_lmetric":14851},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":14851,"score_linear":79495.0,"score_lmetric":94346},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":56286,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":14851,"score_linear":56286.0,"score_lmetric":14851},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":135109,"ongoing_decode_tokens":16057,"pending_prefill_tokens":780,"num_requests":2,"active_p_offloads":0,"cached_blocks":1545,"cache_hit":0,"new_prefill":14851,"score_linear":135109.0,"score_lmetric":31262},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160794,"ongoing_decode_tokens":124906,"pending_prefill_tokens":48,"num_requests":4,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":14851,"score_linear":160794.0,"score_lmetric":59596},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":14851,"score_linear":80246.0,"score_lmetric":14851},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":204169,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45486,"num_requests":2,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":14851,"score_linear":204169.0,"score_lmetric":120674}],"chosen_score_linear":56286.0,"chosen_score_lmetric":14851,"t_first_token":282039.202134567,"t_first_token_unix":1779712904.2160778,"t_done":282039.530276756,"t_done_unix":1779712904.5442197},{"request_id":"1366128:7:1378071:626","session_id":"1366128","input_length":59007,"t_proxy_recv":282039.129559034,"t_decision_unix":1779712904.1435006,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":58368,"affinity_cache_ratio":0.9891707763485688,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":58368,"estimated_new_tokens":639,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":59007,"score_linear":126340.0,"score_lmetric":185347},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1371,"cache_hit":58368,"new_prefill":639,"score_linear":-58368.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":59007,"score_linear":79495.0,"score_lmetric":138502},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71137,"ongoing_decode_tokens":56286,"pending_prefill_tokens":14851,"num_requests":2,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":59007,"score_linear":71137.0,"score_lmetric":147716},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":119052,"ongoing_decode_tokens":119052,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1550,"cache_hit":0,"new_prefill":59007,"score_linear":119052.0,"score_lmetric":59007},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160794,"ongoing_decode_tokens":124906,"pending_prefill_tokens":48,"num_requests":4,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":59007,"score_linear":160794.0,"score_lmetric":236220},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":59007,"score_linear":80246.0,"score_lmetric":59007},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":59007,"score_linear":227626.0,"score_lmetric":314730}],"chosen_score_linear":-58368.0,"chosen_score_lmetric":0,"t_first_token":282039.338742371,"t_first_token_unix":1779712904.352686,"t_done":282040.105930551,"t_done_unix":1779712905.119874},{"request_id":"1269373:22:1358956:522","session_id":"1269373","input_length":80246,"t_proxy_recv":281972.056797542,"t_decision_unix":1779712837.070739,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":78336,"affinity_cache_ratio":0.9761981905640157,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":78336,"estimated_new_tokens":1910,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":7680,"new_prefill":72566,"score_linear":60579.0,"score_lmetric":72566},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":989,"cache_hit":0,"new_prefill":80246,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":80246,"score_linear":150402.0,"score_lmetric":406000},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1321,"cache_hit":7680,"new_prefill":72566,"score_linear":13691.0,"score_lmetric":72566},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":80246,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1230,"cache_hit":0,"new_prefill":80246,"score_linear":83526.0,"score_lmetric":163772},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":78336,"new_prefill":1910,"score_linear":-78336.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":34694,"num_requests":1,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":7680,"new_prefill":72566,"score_linear":51078.0,"score_lmetric":107260}],"chosen_score_linear":-78336.0,"chosen_score_lmetric":0,"t_first_token":281973.096885281,"t_first_token_unix":1779712838.110829,"t_done":282040.226705593,"t_done_unix":1779712905.2406495},{"request_id":"1377061:2:1378590:631","session_id":"1377061","input_length":16515,"t_proxy_recv":282040.900739734,"t_decision_unix":1779712905.9146812,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":15872,"affinity_cache_ratio":0.961065697850439,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":15872,"estimated_new_tokens":643,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":16515,"score_linear":148472.0,"score_lmetric":299254},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1372,"cache_hit":15872,"new_prefill":643,"score_linear":-15872.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":16515,"score_linear":79495.0,"score_lmetric":96010},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":56286,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1637,"cache_hit":0,"new_prefill":16515,"score_linear":56286.0,"score_lmetric":16515},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":16515,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160794,"ongoing_decode_tokens":160794,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":16515,"score_linear":160794.0,"score_lmetric":66060},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":5458,"ongoing_decode_tokens":0,"pending_prefill_tokens":5458,"num_requests":1,"active_p_offloads":0,"cached_blocks":1421,"cache_hit":0,"new_prefill":16515,"score_linear":5458.0,"score_lmetric":21973},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":16515,"score_linear":227626.0,"score_lmetric":187254}],"chosen_score_linear":-15872.0,"chosen_score_lmetric":0,"t_first_token":282040.98461572,"t_first_token_unix":1779712905.9985592,"t_done":282041.055139058,"t_done_unix":1779712906.069082},{"request_id":"1374347:1:1374347:611","session_id":"1374347","input_length":19862,"t_proxy_recv":282026.167139915,"t_decision_unix":1779712891.1810813,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":19862,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":19862,"score_linear":126340.0,"score_lmetric":146202},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":65440,"ongoing_decode_tokens":18676,"pending_prefill_tokens":11948,"num_requests":2,"active_p_offloads":0,"cached_blocks":1290,"cache_hit":0,"new_prefill":19862,"score_linear":65440.0,"score_lmetric":63620},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":19862,"score_linear":79495.0,"score_lmetric":99357},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":0,"pending_prefill_tokens":37854,"num_requests":1,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":19862,"score_linear":56286.0,"score_lmetric":57716},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":117888,"pending_prefill_tokens":2729,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":19862,"score_linear":199977.0,"score_lmetric":67773},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":19862,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1349,"cache_hit":0,"new_prefill":19862,"score_linear":106951.0,"score_lmetric":39724},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181263,"ongoing_decode_tokens":181263,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":19862,"score_linear":181263.0,"score_lmetric":59586}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282028.341842475,"t_first_token_unix":1779712893.355786,"t_done":282041.096848498,"t_done_unix":1779712906.1107914},{"request_id":"1370129:2:1373467:607","session_id":"1370129","input_length":56286,"t_proxy_recv":282023.138135985,"t_decision_unix":1779712888.1520772,"policy":"unified","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":18432,"affinity_cache_ratio":0.3274704189318836,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":18432,"estimated_new_tokens":37854,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":7680,"new_prefill":48606,"score_linear":118660.0,"score_lmetric":174946},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1290,"cache_hit":4096,"new_prefill":52190,"score_linear":-4096.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":56286,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":18432,"new_prefill":37854,"score_linear":-18432.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":56286,"score_linear":199977.0,"score_lmetric":530709},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":151655,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":56286,"score_linear":151655.0,"score_lmetric":168858},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1349,"cache_hit":7680,"new_prefill":48606,"score_linear":99271.0,"score_lmetric":97212},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181263,"ongoing_decode_tokens":181263,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":7680,"new_prefill":48606,"score_linear":173583.0,"score_lmetric":145818}],"chosen_score_linear":-18432.0,"chosen_score_lmetric":0,"t_first_token":282032.835940859,"t_first_token_unix":1779712897.8498845,"t_done":282041.657585845,"t_done_unix":1779712906.6715293},{"request_id":"1378684:1:1378684:632","session_id":"1378684","input_length":7810,"t_proxy_recv":282041.239295884,"t_decision_unix":1779712906.2532372,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7810,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":7810,"score_linear":148472.0,"score_lmetric":281844},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":7810,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":7810,"score_linear":79495.0,"score_lmetric":87305},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":56286,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1637,"cache_hit":0,"new_prefill":7810,"score_linear":56286.0,"score_lmetric":7810},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":7810,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":140932,"ongoing_decode_tokens":140932,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1752,"cache_hit":0,"new_prefill":7810,"score_linear":140932.0,"score_lmetric":23430},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":5458,"ongoing_decode_tokens":5458,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1421,"cache_hit":0,"new_prefill":7810,"score_linear":5458.0,"score_lmetric":7810},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":7810,"score_linear":227626.0,"score_lmetric":161139}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282041.780899893,"t_first_token_unix":1779712906.7948456,"t_done":282042.051895183,"t_done_unix":1779712907.0658386},{"request_id":"1378543:1:1378543:630","session_id":"1378543","input_length":5458,"t_proxy_recv":282040.745989958,"t_decision_unix":1779712905.7599313,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5458,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":5458,"score_linear":148472.0,"score_lmetric":277140},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1372,"cache_hit":0,"new_prefill":5458,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":5458,"score_linear":79495.0,"score_lmetric":84953},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":56286,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1637,"cache_hit":0,"new_prefill":5458,"score_linear":56286.0,"score_lmetric":5458},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":5458,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160794,"ongoing_decode_tokens":160794,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":5458,"score_linear":160794.0,"score_lmetric":21832},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1421,"cache_hit":0,"new_prefill":5458,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":5458,"score_linear":227626.0,"score_lmetric":154083}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282041.095529702,"t_first_token_unix":1779712906.109473,"t_done":282042.170761131,"t_done_unix":1779712907.1847072},{"request_id":"1268861:7:1374579:614","session_id":"1268861","input_length":41508,"t_proxy_recv":282027.066883724,"t_decision_unix":1779712892.080825,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":36864,"affinity_cache_ratio":0.8881179531656548,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":36864,"estimated_new_tokens":4644,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":41508,"score_linear":126340.0,"score_lmetric":167848},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":65440,"ongoing_decode_tokens":18676,"pending_prefill_tokens":11948,"num_requests":2,"active_p_offloads":0,"cached_blocks":1290,"cache_hit":0,"new_prefill":41508,"score_linear":65440.0,"score_lmetric":106912},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":41508,"score_linear":79495.0,"score_lmetric":121003},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":0,"pending_prefill_tokens":37854,"num_requests":1,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":41508,"score_linear":56286.0,"score_lmetric":79362},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":117888,"pending_prefill_tokens":2729,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":21504,"new_prefill":20004,"score_linear":178473.0,"score_lmetric":68199},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83398,"ongoing_decode_tokens":0,"pending_prefill_tokens":20422,"num_requests":2,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":36864,"new_prefill":4644,"score_linear":46534.0,"score_lmetric":50132},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125448,"ongoing_decode_tokens":106951,"pending_prefill_tokens":18497,"num_requests":3,"active_p_offloads":0,"cached_blocks":1349,"cache_hit":0,"new_prefill":41508,"score_linear":125448.0,"score_lmetric":180015},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":158271,"ongoing_decode_tokens":158271,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1399,"cache_hit":0,"new_prefill":41508,"score_linear":158271.0,"score_lmetric":83016}],"chosen_score_linear":46534.0,"chosen_score_lmetric":50132,"t_first_token":282035.533353171,"t_first_token_unix":1779712900.5472965,"t_done":282043.215506622,"t_done_unix":1779712908.2294497},{"request_id":"1379185:1:1379185:634","session_id":"1379185","input_length":1422,"t_proxy_recv":282042.955631438,"t_decision_unix":1779712907.9695725,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1422,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":1422,"score_linear":148472.0,"score_lmetric":269068},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":1422,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":1422,"score_linear":79495.0,"score_lmetric":80917},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":0,"pending_prefill_tokens":435,"num_requests":1,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":1422,"score_linear":15283.0,"score_lmetric":1857},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1566,"cache_hit":0,"new_prefill":1422,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":140932,"ongoing_decode_tokens":140932,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1752,"cache_hit":0,"new_prefill":1422,"score_linear":140932.0,"score_lmetric":4266},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1432,"cache_hit":0,"new_prefill":1422,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":1422,"score_linear":227626.0,"score_lmetric":141975}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282043.054621158,"t_first_token_unix":1779712908.0685644,"t_done":282043.238964123,"t_done_unix":1779712908.252907},{"request_id":"1379569:1:1379569:638","session_id":"1379569","input_length":2632,"t_proxy_recv":282044.185247621,"t_decision_unix":1779712909.199189,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":2632,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":2632,"score_linear":148472.0,"score_lmetric":271488},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":20948,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":2632,"score_linear":20948.0,"score_lmetric":2632},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":2632,"score_linear":79495.0,"score_lmetric":82127},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":2632,"score_linear":134396.0,"score_lmetric":25779},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1568,"cache_hit":0,"new_prefill":2632,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":2632,"score_linear":99424.0,"score_lmetric":5264},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1432,"cache_hit":0,"new_prefill":2632,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":2632,"score_linear":227626.0,"score_lmetric":145605}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282044.348526446,"t_first_token_unix":1779712909.36247,"t_done":282044.7859285,"t_done_unix":1779712909.7998717},{"request_id":"1377061:3:1380174:641","session_id":"1377061","input_length":17368,"t_proxy_recv":282046.307597331,"t_decision_unix":1779712911.3215384,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":16384,"affinity_cache_ratio":0.943344081068632,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":16384,"estimated_new_tokens":984,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":17368,"score_linear":148472.0,"score_lmetric":300960},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":20948,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":16384,"new_prefill":984,"score_linear":4564.0,"score_lmetric":984},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":79495,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":17368,"score_linear":79495.0,"score_lmetric":17368},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":17368,"score_linear":134396.0,"score_lmetric":69987},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1568,"cache_hit":0,"new_prefill":17368,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":17368,"score_linear":99424.0,"score_lmetric":34736},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1437,"cache_hit":0,"new_prefill":17368,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":17368,"score_linear":227626.0,"score_lmetric":189813}],"chosen_score_linear":4564.0,"chosen_score_lmetric":984,"t_first_token":282046.483768571,"t_first_token_unix":1779712911.4977129,"t_done":282046.617016527,"t_done_unix":1779712911.6309602},{"request_id":"1366128:8:1380438:643","session_id":"1366128","input_length":59230,"t_proxy_recv":282047.183842531,"t_decision_unix":1779712912.197784,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9940908323484721,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":350,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":59230,"score_linear":148472.0,"score_lmetric":384684},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":20948,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1374,"cache_hit":58880,"new_prefill":350,"score_linear":-37932.0,"score_lmetric":350},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":79495,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":59230,"score_linear":79495.0,"score_lmetric":59230},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":59230,"score_linear":134396.0,"score_lmetric":195573},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1568,"cache_hit":0,"new_prefill":59230,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":59230,"score_linear":99424.0,"score_lmetric":118460},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":30907,"ongoing_decode_tokens":0,"pending_prefill_tokens":9403,"num_requests":1,"active_p_offloads":0,"cached_blocks":1437,"cache_hit":0,"new_prefill":59230,"score_linear":30907.0,"score_lmetric":68633},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":59230,"score_linear":227626.0,"score_lmetric":315399}],"chosen_score_linear":-37932.0,"chosen_score_lmetric":350,"t_first_token":282047.346572589,"t_first_token_unix":1779712912.360516,"t_done":282048.153290981,"t_done_unix":1779712913.1672347},{"request_id":"1379569:2:1380473:645","session_id":"1379569","input_length":6767,"t_proxy_recv":282047.372301084,"t_decision_unix":1779712912.3862424,"policy":"unified","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":4,"affinity_cache_hit":2560,"affinity_cache_ratio":0.37830648736515443,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":6767,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":6767,"score_linear":148472.0,"score_lmetric":279758},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80178,"ongoing_decode_tokens":80178,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1374,"cache_hit":0,"new_prefill":6767,"score_linear":80178.0,"score_lmetric":13534},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":79495,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":6767,"score_linear":79495.0,"score_lmetric":6767},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":6767,"score_linear":134396.0,"score_lmetric":38184},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1568,"cache_hit":0,"new_prefill":6767,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":6767,"score_linear":99424.0,"score_lmetric":13534},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":36221,"ongoing_decode_tokens":0,"pending_prefill_tokens":9597,"num_requests":2,"active_p_offloads":0,"cached_blocks":1437,"cache_hit":2560,"new_prefill":4207,"score_linear":33661.0,"score_lmetric":27608},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":6767,"score_linear":227626.0,"score_lmetric":158010}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282047.824811778,"t_first_token_unix":1779712912.8387551,"t_done":282048.448067389,"t_done_unix":1779712913.4620106},{"request_id":"1373577:1:1373577:608","session_id":"1373577","input_length":79495,"t_proxy_recv":282023.554929644,"t_decision_unix":1779712888.5688705,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":79495,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":79495,"score_linear":126340.0,"score_lmetric":205835},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1290,"cache_hit":0,"new_prefill":79495,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":79495,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":0,"pending_prefill_tokens":37854,"num_requests":1,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":79495,"score_linear":56286.0,"score_lmetric":117349},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":79495,"score_linear":199977.0,"score_lmetric":600336},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":136154,"ongoing_decode_tokens":136154,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1571,"cache_hit":0,"new_prefill":79495,"score_linear":136154.0,"score_lmetric":158990},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1349,"cache_hit":2560,"new_prefill":76935,"score_linear":104391.0,"score_lmetric":153870},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181263,"ongoing_decode_tokens":181263,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":79495,"score_linear":181263.0,"score_lmetric":238485}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282044.973976474,"t_first_token_unix":1779712909.987925,"t_done":282048.5628023,"t_done_unix":1779712913.576749},{"request_id":"1378543:2:1380447:644","session_id":"1378543","input_length":5314,"t_proxy_recv":282047.225034347,"t_decision_unix":1779712912.238976,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":5120,"affinity_cache_ratio":0.963492660895747,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":5120,"estimated_new_tokens":194,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":5314,"score_linear":148472.0,"score_lmetric":276852},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80178,"ongoing_decode_tokens":20948,"pending_prefill_tokens":350,"num_requests":2,"active_p_offloads":0,"cached_blocks":1374,"cache_hit":0,"new_prefill":5314,"score_linear":80178.0,"score_lmetric":11328},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":79495,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":5314,"score_linear":79495.0,"score_lmetric":5314},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":5314,"score_linear":134396.0,"score_lmetric":33825},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1568,"cache_hit":0,"new_prefill":5314,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":5314,"score_linear":99424.0,"score_lmetric":10628},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":30907,"ongoing_decode_tokens":0,"pending_prefill_tokens":9403,"num_requests":1,"active_p_offloads":0,"cached_blocks":1437,"cache_hit":5120,"new_prefill":194,"score_linear":25787.0,"score_lmetric":9597},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":5314,"score_linear":227626.0,"score_lmetric":153651}],"chosen_score_linear":25787.0,"chosen_score_lmetric":9597,"t_first_token":282048.97220766,"t_first_token_unix":1779712913.9861512,"t_done":282048.972707376,"t_done_unix":1779712913.9866502},{"request_id":"1374465:3:1380413:642","session_id":"1374465","input_length":30907,"t_proxy_recv":282047.099564882,"t_decision_unix":1779712912.1135063,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":21504,"affinity_cache_ratio":0.6957647134953246,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":21504,"estimated_new_tokens":9403,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":30907,"score_linear":148472.0,"score_lmetric":328038},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":20948,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1374,"cache_hit":0,"new_prefill":30907,"score_linear":20948.0,"score_lmetric":30907},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":79495,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":30907,"score_linear":79495.0,"score_lmetric":30907},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":30907,"score_linear":134396.0,"score_lmetric":110604},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1568,"cache_hit":0,"new_prefill":30907,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":30907,"score_linear":99424.0,"score_lmetric":61814},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1437,"cache_hit":21504,"new_prefill":9403,"score_linear":-21504.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":30907,"score_linear":227626.0,"score_lmetric":230430}],"chosen_score_linear":-21504.0,"chosen_score_lmetric":0,"t_first_token":282048.967240596,"t_first_token_unix":1779712913.9811845,"t_done":282049.60569145,"t_done_unix":1779712914.6196346},{"request_id":"1363943:8:1379215:635","session_id":"1363943","input_length":20948,"t_proxy_recv":282043.110105601,"t_decision_unix":1779712908.1240468,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":20480,"affinity_cache_ratio":0.97765896505633,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":20480,"estimated_new_tokens":468,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":8704,"new_prefill":12244,"score_linear":139768.0,"score_lmetric":290712},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":20480,"new_prefill":468,"score_linear":-20480.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":20948,"score_linear":79495.0,"score_lmetric":100443},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":0,"pending_prefill_tokens":435,"num_requests":1,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":4096,"new_prefill":16852,"score_linear":11187.0,"score_lmetric":17287},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":1422,"ongoing_decode_tokens":1422,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1566,"cache_hit":0,"new_prefill":20948,"score_linear":1422.0,"score_lmetric":20948},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":140932,"ongoing_decode_tokens":140932,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1752,"cache_hit":0,"new_prefill":20948,"score_linear":140932.0,"score_lmetric":62844},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1432,"cache_hit":4096,"new_prefill":16852,"score_linear":-4096.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":4096,"new_prefill":16852,"score_linear":223530.0,"score_lmetric":188265}],"chosen_score_linear":-20480.0,"chosen_score_lmetric":0,"t_first_token":282043.161211352,"t_first_token_unix":1779712908.1751547,"t_done":282050.301734241,"t_done_unix":1779712915.3156781},{"request_id":"1379569:3:1381510:649","session_id":"1379569","input_length":7620,"t_proxy_recv":282050.998426433,"t_decision_unix":1779712916.0123677,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":6656,"affinity_cache_ratio":0.873490813648294,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":6656,"estimated_new_tokens":964,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":7620,"score_linear":148472.0,"score_lmetric":281464},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":13989,"ongoing_decode_tokens":0,"pending_prefill_tokens":13989,"num_requests":1,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":0,"new_prefill":7620,"score_linear":13989.0,"score_lmetric":21609},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":81147,"ongoing_decode_tokens":81147,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":7620,"score_linear":81147.0,"score_lmetric":7620},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":7620,"score_linear":134396.0,"score_lmetric":40743},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1581,"cache_hit":6656,"new_prefill":964,"score_linear":-6656.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":7620,"score_linear":99424.0,"score_lmetric":15240},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1455,"cache_hit":2560,"new_prefill":5060,"score_linear":-2560.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":7620,"score_linear":227626.0,"score_lmetric":160569}],"chosen_score_linear":-6656.0,"chosen_score_lmetric":0,"t_first_token":282051.106857166,"t_first_token_unix":1779712916.1208005,"t_done":282051.617502867,"t_done_unix":1779712916.6314461},{"request_id":"1362265:4:1381673:650","session_id":"1362265","input_length":10645,"t_proxy_recv":282051.681425458,"t_decision_unix":1779712916.6953669,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":10240,"affinity_cache_ratio":0.9619539689995303,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":10240,"estimated_new_tokens":405,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":6656,"new_prefill":3989,"score_linear":141816.0,"score_lmetric":274202},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":13989,"ongoing_decode_tokens":0,"pending_prefill_tokens":13989,"num_requests":1,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":10240,"new_prefill":405,"score_linear":3749.0,"score_lmetric":14394},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":81147,"ongoing_decode_tokens":81147,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":10645,"score_linear":81147.0,"score_lmetric":10645},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":6656,"new_prefill":3989,"score_linear":127740.0,"score_lmetric":29850},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":10645,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":10645,"score_linear":99424.0,"score_lmetric":21290},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1455,"cache_hit":0,"new_prefill":10645,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":10645,"score_linear":227626.0,"score_lmetric":169644}],"chosen_score_linear":3749.0,"chosen_score_lmetric":14394,"t_first_token":282051.8626766,"t_first_token_unix":1779712916.8766198,"t_done":282052.15189669,"t_done_unix":1779712917.1658404},{"request_id":"1381728:1:1381728:651","session_id":"1381728","input_length":1391,"t_proxy_recv":282051.903996036,"t_decision_unix":1779712916.9179378,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1391,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":1391,"score_linear":148472.0,"score_lmetric":269006},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":24634,"ongoing_decode_tokens":24634,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":0,"new_prefill":1391,"score_linear":24634.0,"score_lmetric":2782},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":81147,"ongoing_decode_tokens":81147,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":1391,"score_linear":81147.0,"score_lmetric":1391},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":1391,"score_linear":134396.0,"score_lmetric":22056},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":1391,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":1391,"score_linear":99424.0,"score_lmetric":2782},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1455,"cache_hit":0,"new_prefill":1391,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":1391,"score_linear":227626.0,"score_lmetric":141882}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282052.007434459,"t_first_token_unix":1779712917.021378,"t_done":282052.154951485,"t_done_unix":1779712917.1688948},{"request_id":"1373577:2:1380740:647","session_id":"1373577","input_length":81147,"t_proxy_recv":282048.5855165,"t_decision_unix":1779712913.5994575,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":79872,"affinity_cache_ratio":0.9842877740397057,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":79872,"estimated_new_tokens":1275,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":81147,"score_linear":148472.0,"score_lmetric":428518},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":20948,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1374,"cache_hit":0,"new_prefill":81147,"score_linear":20948.0,"score_lmetric":81147},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":79872,"new_prefill":1275,"score_linear":-79872.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":81147,"score_linear":134396.0,"score_lmetric":261324},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1581,"cache_hit":0,"new_prefill":81147,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":81147,"score_linear":99424.0,"score_lmetric":162294},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":36221,"ongoing_decode_tokens":0,"pending_prefill_tokens":9597,"num_requests":2,"active_p_offloads":0,"cached_blocks":1437,"cache_hit":2560,"new_prefill":78587,"score_linear":33661.0,"score_lmetric":176368},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":81147,"score_linear":227626.0,"score_lmetric":381150}],"chosen_score_linear":-79872.0,"chosen_score_lmetric":0,"t_first_token":282049.446133318,"t_first_token_unix":1779712914.460077,"t_done":282052.615102951,"t_done_unix":1779712917.6290472},{"request_id":"1366128:9:1381799:652","session_id":"1366128","input_length":59374,"t_proxy_recv":282052.164191519,"t_decision_unix":1779712917.1781328,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9916798598713241,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":494,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":59374,"score_linear":148472.0,"score_lmetric":384972},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":13989,"ongoing_decode_tokens":13989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":58880,"new_prefill":494,"score_linear":-44891.0,"score_lmetric":494},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":81147,"ongoing_decode_tokens":81147,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":59374,"score_linear":81147.0,"score_lmetric":59374},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":59374,"score_linear":134396.0,"score_lmetric":196005},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":59374,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":59374,"score_linear":99424.0,"score_lmetric":118748},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":59374,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":59374,"score_linear":227626.0,"score_lmetric":315831}],"chosen_score_linear":-44891.0,"chosen_score_lmetric":494,"t_first_token":282052.303612029,"t_first_token_unix":1779712917.3175557,"t_done":282052.756431808,"t_done_unix":1779712917.7703753},{"request_id":"1381387:1:1381387:648","session_id":"1381387","input_length":13989,"t_proxy_recv":282050.586442833,"t_decision_unix":1779712915.6003838,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":13989,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":13989,"score_linear":148472.0,"score_lmetric":294202},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":0,"new_prefill":13989,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":81147,"ongoing_decode_tokens":81147,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":13989,"score_linear":81147.0,"score_lmetric":13989},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":13989,"score_linear":134396.0,"score_lmetric":59850},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1581,"cache_hit":0,"new_prefill":13989,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":13989,"score_linear":99424.0,"score_lmetric":27978},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1455,"cache_hit":0,"new_prefill":13989,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":13989,"score_linear":227626.0,"score_lmetric":179676}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282051.792256303,"t_first_token_unix":1779712916.8061998,"t_done":282053.719391145,"t_done_unix":1779712918.7333345},{"request_id":"1382294:1:1382294:656","session_id":"1382294","input_length":3034,"t_proxy_recv":282053.907119382,"t_decision_unix":1779712918.9210606,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":3034,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":3034,"score_linear":148472.0,"score_lmetric":272292},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":3034,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38111,"ongoing_decode_tokens":0,"pending_prefill_tokens":38111,"num_requests":1,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":3034,"score_linear":38111.0,"score_lmetric":41145},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":3034,"score_linear":134396.0,"score_lmetric":26985},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":54012,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":1,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":3034,"score_linear":54012.0,"score_lmetric":57046},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":3034,"score_linear":99424.0,"score_lmetric":6068},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":0,"pending_prefill_tokens":19753,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":3034,"score_linear":47401.0,"score_lmetric":22787},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":3034,"score_linear":227626.0,"score_lmetric":146811}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282054.09394001,"t_first_token_unix":1779712919.107883,"t_done":282054.251531797,"t_done_unix":1779712919.265475},{"request_id":"1382710:1:1382710:659","session_id":"1382710","input_length":3190,"t_proxy_recv":282055.423382215,"t_decision_unix":1779712920.4373238,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":5145,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":3190,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":3190,"score_linear":148472.0,"score_lmetric":272604},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":1955,"ongoing_decode_tokens":0,"pending_prefill_tokens":1955,"num_requests":1,"active_p_offloads":0,"cached_blocks":1409,"cache_hit":0,"new_prefill":3190,"score_linear":1955.0,"score_lmetric":5145},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38111,"ongoing_decode_tokens":0,"pending_prefill_tokens":38111,"num_requests":1,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":3190,"score_linear":38111.0,"score_lmetric":41301},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":3190,"score_linear":134396.0,"score_lmetric":27453},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":54012,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":1,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":3190,"score_linear":54012.0,"score_lmetric":57202},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":3190,"score_linear":99424.0,"score_lmetric":6380},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":0,"pending_prefill_tokens":19753,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":3190,"score_linear":47401.0,"score_lmetric":22943},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":3190,"score_linear":227626.0,"score_lmetric":147279}],"chosen_score_linear":1955.0,"chosen_score_lmetric":5145,"t_first_token":282055.63726814,"t_first_token_unix":1779712920.6512113,"t_done":282055.809896475,"t_done_unix":1779712920.8238394},{"request_id":"1382667:1:1382667:658","session_id":"1382667","input_length":1955,"t_proxy_recv":282055.322762257,"t_decision_unix":1779712920.3367035,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":1955,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":1955,"score_linear":148472.0,"score_lmetric":270134},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1409,"cache_hit":0,"new_prefill":1955,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38111,"ongoing_decode_tokens":0,"pending_prefill_tokens":38111,"num_requests":1,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":1955,"score_linear":38111.0,"score_lmetric":40066},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":1955,"score_linear":134396.0,"score_lmetric":23748},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":54012,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":1,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":1955,"score_linear":54012.0,"score_lmetric":55967},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":1955,"score_linear":99424.0,"score_lmetric":3910},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":0,"pending_prefill_tokens":19753,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":1955,"score_linear":47401.0,"score_lmetric":21708},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":1955,"score_linear":227626.0,"score_lmetric":143574}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282055.447849816,"t_first_token_unix":1779712920.461793,"t_done":282056.024357782,"t_done_unix":1779712921.0383008},{"request_id":"1382804:1:1382804:660","session_id":"1382804","input_length":627,"t_proxy_recv":282055.650808919,"t_decision_unix":1779712920.66475,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":1254,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":627,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":627,"score_linear":148472.0,"score_lmetric":267478},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":5145,"ongoing_decode_tokens":5145,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1409,"cache_hit":0,"new_prefill":627,"score_linear":5145.0,"score_lmetric":1254},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38111,"ongoing_decode_tokens":0,"pending_prefill_tokens":38111,"num_requests":1,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":627,"score_linear":38111.0,"score_lmetric":38738},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":627,"score_linear":134396.0,"score_lmetric":19764},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":54012,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":1,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":627,"score_linear":54012.0,"score_lmetric":54639},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":627,"score_linear":99424.0,"score_lmetric":1254},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":0,"pending_prefill_tokens":19753,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":627,"score_linear":47401.0,"score_lmetric":20380},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":627,"score_linear":227626.0,"score_lmetric":139590}],"chosen_score_linear":99424.0,"chosen_score_lmetric":1254,"t_first_token":282055.901881146,"t_first_token_unix":1779712920.9158242,"t_done":282056.32531013,"t_done_unix":1779712921.3392532},{"request_id":"1382294:2:1383073:664","session_id":"1382294","input_length":5164,"t_proxy_recv":282056.518352079,"t_decision_unix":1779712921.5322933,"policy":"unified","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":5,"affinity_cache_hit":2560,"affinity_cache_ratio":0.4957397366382649,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":10328,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":5164,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":5164,"score_linear":148472.0,"score_lmetric":276552},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":64641,"ongoing_decode_tokens":59506,"pending_prefill_tokens":5135,"num_requests":2,"active_p_offloads":0,"cached_blocks":1418,"cache_hit":2560,"new_prefill":2604,"score_linear":62081.0,"score_lmetric":15478},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38111,"ongoing_decode_tokens":0,"pending_prefill_tokens":38111,"num_requests":1,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":5164,"score_linear":38111.0,"score_lmetric":43275},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":5164,"score_linear":134396.0,"score_lmetric":33375},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":54012,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":1,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":5164,"score_linear":54012.0,"score_lmetric":59176},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":5164,"score_linear":99424.0,"score_lmetric":10328},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":0,"pending_prefill_tokens":19753,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":5164,"score_linear":47401.0,"score_lmetric":24917},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":5164,"score_linear":227626.0,"score_lmetric":153201}],"chosen_score_linear":99424.0,"chosen_score_lmetric":10328,"t_first_token":282056.889234425,"t_first_token_unix":1779712921.9031775,"t_done":282057.49119476,"t_done_unix":1779712922.505138},{"request_id":"1366128:10:1382843:661","session_id":"1366128","input_length":59506,"t_proxy_recv":282055.776594962,"t_decision_unix":1779712920.7905352,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":59392,"affinity_cache_ratio":0.9980842268006588,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":59392,"estimated_new_tokens":114,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":59506,"score_linear":148472.0,"score_lmetric":385236},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":5145,"ongoing_decode_tokens":5145,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1409,"cache_hit":59392,"new_prefill":114,"score_linear":-54247.0,"score_lmetric":228},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38111,"ongoing_decode_tokens":0,"pending_prefill_tokens":38111,"num_requests":1,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":59506,"score_linear":38111.0,"score_lmetric":97617},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":59506,"score_linear":134396.0,"score_lmetric":196401},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":54012,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":1,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":59506,"score_linear":54012.0,"score_lmetric":113518},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":100051,"ongoing_decode_tokens":99424,"pending_prefill_tokens":627,"num_requests":3,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":59506,"score_linear":100051.0,"score_lmetric":180399},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":0,"pending_prefill_tokens":19753,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":59506,"score_linear":47401.0,"score_lmetric":79259},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":59506,"score_linear":227626.0,"score_lmetric":316227}],"chosen_score_linear":-54247.0,"chosen_score_lmetric":228,"t_first_token":282055.90419049,"t_first_token_unix":1779712920.9181333,"t_done":282057.502421088,"t_done_unix":1779712922.5163643},{"request_id":"1383090:1:1383090:665","session_id":"1383090","input_length":9305,"t_proxy_recv":282056.562758806,"t_decision_unix":1779712921.5767002,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":28880,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":9305,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":9305,"score_linear":148472.0,"score_lmetric":284834},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":64641,"ongoing_decode_tokens":59506,"pending_prefill_tokens":5135,"num_requests":2,"active_p_offloads":0,"cached_blocks":1418,"cache_hit":0,"new_prefill":9305,"score_linear":64641.0,"score_lmetric":28880},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38111,"ongoing_decode_tokens":0,"pending_prefill_tokens":38111,"num_requests":1,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":9305,"score_linear":38111.0,"score_lmetric":47416},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":9305,"score_linear":134396.0,"score_lmetric":45798},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":54012,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":1,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":9305,"score_linear":54012.0,"score_lmetric":63317},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104588,"ongoing_decode_tokens":99424,"pending_prefill_tokens":5164,"num_requests":3,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":9305,"score_linear":104588.0,"score_lmetric":43407},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":0,"pending_prefill_tokens":19753,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":9305,"score_linear":47401.0,"score_lmetric":29058},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":9305,"score_linear":227626.0,"score_lmetric":165624}],"chosen_score_linear":64641.0,"chosen_score_lmetric":28880,"t_first_token":282057.395598193,"t_first_token_unix":1779712922.4095416,"t_done":282057.723756002,"t_done_unix":1779712922.7376995},{"request_id":"1383009:1:1383009:663","session_id":"1383009","input_length":5135,"t_proxy_recv":282056.295192567,"t_decision_unix":1779712921.3091338,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":5135,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":5135,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":5135,"score_linear":148472.0,"score_lmetric":276494},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":59506,"ongoing_decode_tokens":59506,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1418,"cache_hit":0,"new_prefill":5135,"score_linear":59506.0,"score_lmetric":5135},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38111,"ongoing_decode_tokens":0,"pending_prefill_tokens":38111,"num_requests":1,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":5135,"score_linear":38111.0,"score_lmetric":43246},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":5135,"score_linear":134396.0,"score_lmetric":33288},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":54012,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":1,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":5135,"score_linear":54012.0,"score_lmetric":59147},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":100051,"ongoing_decode_tokens":100051,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":5135,"score_linear":100051.0,"score_lmetric":15405},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":0,"pending_prefill_tokens":19753,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":5135,"score_linear":47401.0,"score_lmetric":24888},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":5135,"score_linear":227626.0,"score_lmetric":153114}],"chosen_score_linear":59506.0,"chosen_score_lmetric":5135,"t_first_token":282056.654911647,"t_first_token_unix":1779712921.6688547,"t_done":282057.744298934,"t_done_unix":1779712922.7582424},{"request_id":"1263749:2:1379302:636","session_id":"1263749","input_length":57656,"t_proxy_recv":282043.435183826,"t_decision_unix":1779712908.449125,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":56320,"affinity_cache_ratio":0.9768280838074095,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":56320,"estimated_new_tokens":1336,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":57656,"score_linear":148472.0,"score_lmetric":381536},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":20948,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":57656,"score_linear":20948.0,"score_lmetric":57656},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":57656,"score_linear":79495.0,"score_lmetric":137151},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":0,"pending_prefill_tokens":435,"num_requests":1,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":56320,"new_prefill":1336,"score_linear":-41037.0,"score_lmetric":1771},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1568,"cache_hit":0,"new_prefill":57656,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":57656,"score_linear":99424.0,"score_lmetric":115312},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1432,"cache_hit":0,"new_prefill":57656,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":57656,"score_linear":227626.0,"score_lmetric":310677}],"chosen_score_linear":-41037.0,"chosen_score_lmetric":1771,"t_first_token":282057.289175943,"t_first_token_unix":1779712922.3031197,"t_done":282057.989585922,"t_done_unix":1779712923.0035293},{"request_id":"1383625:1:1383625:668","session_id":"1383625","input_length":3186,"t_proxy_recv":282058.492489703,"t_decision_unix":1779712923.5064309,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":3186,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":3186,"score_linear":148472.0,"score_lmetric":272596},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1446,"cache_hit":0,"new_prefill":3186,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38111,"ongoing_decode_tokens":0,"pending_prefill_tokens":38111,"num_requests":1,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":3186,"score_linear":38111.0,"score_lmetric":41297},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76740,"ongoing_decode_tokens":76740,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":3186,"score_linear":76740.0,"score_lmetric":6372},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":64229,"ongoing_decode_tokens":0,"pending_prefill_tokens":56549,"num_requests":2,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":3186,"score_linear":64229.0,"score_lmetric":119470},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1772,"cache_hit":0,"new_prefill":3186,"score_linear":99424.0,"score_lmetric":6372},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":47401,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":3186,"score_linear":47401.0,"score_lmetric":3186},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":3186,"score_linear":227626.0,"score_lmetric":147267}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282058.688112459,"t_first_token_unix":1779712923.7020564,"t_done":282058.82453689,"t_done_unix":1779712923.8384802},{"request_id":"1382089:1:1382089:655","session_id":"1382089","input_length":38111,"t_proxy_recv":282053.17660026,"t_decision_unix":1779712918.1905413,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":38111,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":38111,"score_linear":148472.0,"score_lmetric":342446},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":13989,"ongoing_decode_tokens":13989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1377,"cache_hit":0,"new_prefill":38111,"score_linear":13989.0,"score_lmetric":38111},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":38111,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":38111,"score_linear":134396.0,"score_lmetric":132216},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":38111,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":38111,"score_linear":99424.0,"score_lmetric":76222},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":0,"pending_prefill_tokens":19753,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":38111,"score_linear":47401.0,"score_lmetric":57864},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":34816,"new_prefill":3295,"score_linear":192810.0,"score_lmetric":147594}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282059.043148322,"t_first_token_unix":1779712924.0570924,"t_done":282059.185033903,"t_done_unix":1779712924.198985},{"request_id":"1382667:2:1384299:674","session_id":"1382667","input_length":5989,"t_proxy_recv":282060.975347362,"t_decision_unix":1779712925.9892883,"policy":"unified","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":2,"affinity_cache_hit":1536,"affinity_cache_ratio":0.2564701953581566,"affinity_num_requests":3,"avg_num_requests":2.125,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":5989,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":5989,"score_linear":148472.0,"score_lmetric":278202},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103787,"ongoing_decode_tokens":0,"pending_prefill_tokens":27499,"num_requests":3,"active_p_offloads":0,"cached_blocks":1452,"cache_hit":1536,"new_prefill":4453,"score_linear":102251.0,"score_lmetric":95856},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1818,"cache_hit":0,"new_prefill":5989,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76740,"ongoing_decode_tokens":76740,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":5989,"score_linear":76740.0,"score_lmetric":11978},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":133694,"ongoing_decode_tokens":0,"pending_prefill_tokens":59966,"num_requests":3,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":5989,"score_linear":133694.0,"score_lmetric":197865},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1772,"cache_hit":0,"new_prefill":5989,"score_linear":99424.0,"score_lmetric":11978},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":68622,"ongoing_decode_tokens":47401,"pending_prefill_tokens":229,"num_requests":2,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":5989,"score_linear":68622.0,"score_lmetric":12436},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":5989,"score_linear":227626.0,"score_lmetric":155676}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282061.364319494,"t_first_token_unix":1779712926.3782628,"t_done":282061.672390642,"t_done_unix":1779712926.6863337},{"request_id":"1363093:3:1374394:612","session_id":"1363093","input_length":63536,"t_proxy_recv":282026.321736138,"t_decision_unix":1779712891.3356771,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":62976,"affinity_cache_ratio":0.9911860992193402,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":62976,"estimated_new_tokens":560,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":63536,"score_linear":126340.0,"score_lmetric":189876},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":65440,"ongoing_decode_tokens":18676,"pending_prefill_tokens":11948,"num_requests":2,"active_p_offloads":0,"cached_blocks":1290,"cache_hit":0,"new_prefill":63536,"score_linear":65440.0,"score_lmetric":150968},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":63536,"score_linear":79495.0,"score_lmetric":143031},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":0,"pending_prefill_tokens":37854,"num_requests":1,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":63536,"score_linear":56286.0,"score_lmetric":101390},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":117888,"pending_prefill_tokens":2729,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":63536,"score_linear":199977.0,"score_lmetric":198795},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":19862,"ongoing_decode_tokens":0,"pending_prefill_tokens":19862,"num_requests":1,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":62976,"new_prefill":560,"score_linear":-43114.0,"score_lmetric":20422},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1349,"cache_hit":0,"new_prefill":63536,"score_linear":106951.0,"score_lmetric":127072},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181263,"ongoing_decode_tokens":181263,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":63536,"score_linear":181263.0,"score_lmetric":190608}],"chosen_score_linear":-43114.0,"chosen_score_lmetric":20422,"t_first_token":282028.344078889,"t_first_token_unix":1779712893.3580225,"t_done":282062.394778036,"t_done_unix":1779712927.4087212},{"request_id":"1384772:1:1384772:677","session_id":"1384772","input_length":4690,"t_proxy_recv":282062.439177667,"t_decision_unix":1779712927.4531193,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":4690,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":0,"pending_prefill_tokens":133313,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":4690,"score_linear":198337.0,"score_lmetric":414009},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103787,"ongoing_decode_tokens":28383,"pending_prefill_tokens":12940,"num_requests":3,"active_p_offloads":0,"cached_blocks":1452,"cache_hit":0,"new_prefill":4690,"score_linear":103787.0,"score_lmetric":52890},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1829,"cache_hit":0,"new_prefill":4690,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76740,"ongoing_decode_tokens":76740,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":4690,"score_linear":76740.0,"score_lmetric":9380},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":133694,"ongoing_decode_tokens":0,"pending_prefill_tokens":59966,"num_requests":3,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":4690,"score_linear":133694.0,"score_lmetric":193968},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":4690,"score_linear":35888.0,"score_lmetric":4690},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":68622,"ongoing_decode_tokens":47401,"pending_prefill_tokens":229,"num_requests":2,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":4690,"score_linear":68622.0,"score_lmetric":9838},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":227626,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":4690,"score_linear":227626.0,"score_lmetric":14070}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282062.731210741,"t_first_token_unix":1779712927.745154,"t_done":282062.864787743,"t_done_unix":1779712927.878731},{"request_id":"1370129:3:1379538:637","session_id":"1370129","input_length":61457,"t_proxy_recv":282044.077327085,"t_decision_unix":1779712909.0912683,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":56832,"affinity_cache_ratio":0.9247441300421433,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":56832,"estimated_new_tokens":4625,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":7680,"new_prefill":53777,"score_linear":140792.0,"score_lmetric":373778},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":20948,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":4096,"new_prefill":57361,"score_linear":16852.0,"score_lmetric":57361},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":61457,"score_linear":79495.0,"score_lmetric":140952},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72939,"ongoing_decode_tokens":15283,"pending_prefill_tokens":1336,"num_requests":2,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":56832,"new_prefill":4625,"score_linear":16107.0,"score_lmetric":11922},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1568,"cache_hit":0,"new_prefill":61457,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":61457,"score_linear":99424.0,"score_lmetric":122914},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1432,"cache_hit":7680,"new_prefill":53777,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":7680,"new_prefill":53777,"score_linear":219946.0,"score_lmetric":299040}],"chosen_score_linear":16107.0,"chosen_score_lmetric":11922,"t_first_token":282057.294693737,"t_first_token_unix":1779712922.3086398,"t_done":282063.774129609,"t_done_unix":1779712928.788073},{"request_id":"1383009:2:1384930:679","session_id":"1383009","input_length":11113,"t_proxy_recv":282062.946784786,"t_decision_unix":1779712927.960726,"policy":"unified","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":2,"affinity_cache_hit":5120,"affinity_cache_ratio":0.46072167731485647,"affinity_num_requests":3,"avg_num_requests":2.125,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":11113,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":0,"pending_prefill_tokens":133313,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":11113,"score_linear":198337.0,"score_lmetric":433278},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103787,"ongoing_decode_tokens":28383,"pending_prefill_tokens":12940,"num_requests":3,"active_p_offloads":0,"cached_blocks":1452,"cache_hit":5120,"new_prefill":5993,"score_linear":98667.0,"score_lmetric":56799},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1838,"cache_hit":0,"new_prefill":11113,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76740,"ongoing_decode_tokens":76740,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":11113,"score_linear":76740.0,"score_lmetric":22226},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":133694,"ongoing_decode_tokens":0,"pending_prefill_tokens":59966,"num_requests":3,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":11113,"score_linear":133694.0,"score_lmetric":213237},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":11113,"score_linear":35888.0,"score_lmetric":11113},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":68622,"ongoing_decode_tokens":68622,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":11113,"score_linear":68622.0,"score_lmetric":22226},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":227626,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":11113,"score_linear":227626.0,"score_lmetric":33339}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282063.827641493,"t_first_token_unix":1779712928.841585,"t_done":282063.991527762,"t_done_unix":1779712929.005471},{"request_id":"1340290:3:1384149:672","session_id":"1340290","input_length":21221,"t_proxy_recv":282060.442490056,"t_decision_unix":1779712925.4564312,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":20992,"affinity_cache_ratio":0.9892088026011969,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":20992,"estimated_new_tokens":229,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":21221,"score_linear":148472.0,"score_lmetric":308666},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99637,"ongoing_decode_tokens":0,"pending_prefill_tokens":26421,"num_requests":2,"active_p_offloads":0,"cached_blocks":1452,"cache_hit":0,"new_prefill":21221,"score_linear":99637.0,"score_lmetric":95284},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1818,"cache_hit":0,"new_prefill":21221,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76740,"ongoing_decode_tokens":76740,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":21221,"score_linear":76740.0,"score_lmetric":42442},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":133694,"ongoing_decode_tokens":0,"pending_prefill_tokens":59966,"num_requests":3,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":21221,"score_linear":133694.0,"score_lmetric":243561},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1772,"cache_hit":0,"new_prefill":21221,"score_linear":99424.0,"score_lmetric":42442},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":47401,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":20992,"new_prefill":229,"score_linear":26409.0,"score_lmetric":229},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":21221,"score_linear":227626.0,"score_lmetric":201372}],"chosen_score_linear":26409.0,"chosen_score_lmetric":229,"t_first_token":282062.709596814,"t_first_token_unix":1779712927.7235403,"t_done":282064.560342236,"t_done_unix":1779712929.5742853},{"request_id":"1355484:8:1377993:625","session_id":"1355484","input_length":23457,"t_proxy_recv":282038.851640692,"t_decision_unix":1779712903.8655818,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9822227906381891,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":23040,"estimated_new_tokens":417,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":23457,"score_linear":126340.0,"score_lmetric":149797},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1371,"cache_hit":0,"new_prefill":23457,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":23457,"score_linear":79495.0,"score_lmetric":102952},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71137,"ongoing_decode_tokens":56286,"pending_prefill_tokens":14851,"num_requests":2,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":23457,"score_linear":71137.0,"score_lmetric":76616},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":119052,"ongoing_decode_tokens":119052,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1550,"cache_hit":0,"new_prefill":23457,"score_linear":119052.0,"score_lmetric":23457},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160794,"ongoing_decode_tokens":124906,"pending_prefill_tokens":48,"num_requests":4,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":23457,"score_linear":160794.0,"score_lmetric":94020},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":23457,"score_linear":80246.0,"score_lmetric":23457},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":204169,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45486,"num_requests":2,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":23040,"new_prefill":417,"score_linear":181129.0,"score_lmetric":91806}],"chosen_score_linear":181129.0,"chosen_score_lmetric":91806,"t_first_token":282061.033252661,"t_first_token_unix":1779712926.047196,"t_done":282065.236851225,"t_done_unix":1779712930.2507946},{"request_id":"1384947:1:1384947:680","session_id":"1384947","input_length":19034,"t_proxy_recv":282063.006730769,"t_decision_unix":1779712928.020672,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.25,"fallback_score":19034,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":19034,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":0,"pending_prefill_tokens":133313,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":19034,"score_linear":198337.0,"score_lmetric":457041},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103787,"ongoing_decode_tokens":28383,"pending_prefill_tokens":12940,"num_requests":3,"active_p_offloads":0,"cached_blocks":1452,"cache_hit":0,"new_prefill":19034,"score_linear":103787.0,"score_lmetric":95922},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":11113,"ongoing_decode_tokens":0,"pending_prefill_tokens":11113,"num_requests":1,"active_p_offloads":0,"cached_blocks":1838,"cache_hit":0,"new_prefill":19034,"score_linear":11113.0,"score_lmetric":30147},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76740,"ongoing_decode_tokens":76740,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":19034,"score_linear":76740.0,"score_lmetric":38068},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":133694,"ongoing_decode_tokens":0,"pending_prefill_tokens":59966,"num_requests":3,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":19034,"score_linear":133694.0,"score_lmetric":237000},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":19034,"score_linear":35888.0,"score_lmetric":19034},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":68622,"ongoing_decode_tokens":68622,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":19034,"score_linear":68622.0,"score_lmetric":38068},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":227626,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":19034,"score_linear":227626.0,"score_lmetric":57102}],"chosen_score_linear":35888.0,"chosen_score_lmetric":19034,"t_first_token":282064.940577093,"t_first_token_unix":1779712929.954521,"t_done":282065.323391188,"t_done_unix":1779712930.3373342},{"request_id":"1301929:2:1375154:617","session_id":"1301929","input_length":35888,"t_proxy_recv":282029.054334324,"t_decision_unix":1779712894.0682755,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":35840,"affinity_cache_ratio":0.9986625055728935,"affinity_num_requests":3,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":35840,"estimated_new_tokens":48,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":3584,"new_prefill":32304,"score_linear":122756.0,"score_lmetric":158644},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":58511,"ongoing_decode_tokens":0,"pending_prefill_tokens":11919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":35888,"score_linear":58511.0,"score_lmetric":47807},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":35888,"score_linear":79495.0,"score_lmetric":115383},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":0,"pending_prefill_tokens":37854,"num_requests":1,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":35888,"score_linear":56286.0,"score_lmetric":73742},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":183920,"pending_prefill_tokens":2233,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":35888,"score_linear":199977.0,"score_lmetric":114363},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124906,"ongoing_decode_tokens":83398,"pending_prefill_tokens":4644,"num_requests":3,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":35840,"new_prefill":48,"score_linear":89066.0,"score_lmetric":14076},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":35888,"score_linear":106951.0,"score_lmetric":71776},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":158271,"ongoing_decode_tokens":158271,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1399,"cache_hit":0,"new_prefill":35888,"score_linear":158271.0,"score_lmetric":71776}],"chosen_score_linear":89066.0,"chosen_score_lmetric":14076,"t_first_token":282040.290140991,"t_first_token_unix":1779712905.304085,"t_done":282066.008336752,"t_done_unix":1779712931.02228},{"request_id":"1379569:4:1383116:666","session_id":"1379569","input_length":10217,"t_proxy_recv":282056.651822728,"t_decision_unix":1779712921.6657639,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":7680,"affinity_cache_ratio":0.7516883625330332,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":2537,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":10217,"score_linear":148472.0,"score_lmetric":286658},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":73946,"ongoing_decode_tokens":59506,"pending_prefill_tokens":14440,"num_requests":3,"active_p_offloads":0,"cached_blocks":1418,"cache_hit":0,"new_prefill":10217,"score_linear":73946.0,"score_lmetric":73971},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38111,"ongoing_decode_tokens":0,"pending_prefill_tokens":38111,"num_requests":1,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":10217,"score_linear":38111.0,"score_lmetric":48328},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":10217,"score_linear":134396.0,"score_lmetric":48534},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":54012,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":1,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":7680,"new_prefill":2537,"score_linear":46332.0,"score_lmetric":56549},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104588,"ongoing_decode_tokens":99424,"pending_prefill_tokens":5164,"num_requests":3,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":10217,"score_linear":104588.0,"score_lmetric":46143},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":0,"pending_prefill_tokens":19753,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":2560,"new_prefill":7657,"score_linear":44841.0,"score_lmetric":27410},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":10217,"score_linear":227626.0,"score_lmetric":168360}],"chosen_score_linear":46332.0,"chosen_score_lmetric":56549,"t_first_token":282064.129450238,"t_first_token_unix":1779712929.1433933,"t_done":282066.574893317,"t_done_unix":1779712931.5888364},{"request_id":"1340290:4:1385568:684","session_id":"1340290","input_length":26146,"t_proxy_recv":282065.303741298,"t_decision_unix":1779712930.317683,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":20992,"affinity_cache_ratio":0.8028761569647365,"affinity_num_requests":1,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"cache_hit":20992,"estimated_new_tokens":5154,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":0,"pending_prefill_tokens":133313,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":26146,"score_linear":198337.0,"score_lmetric":478377},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120521,"ongoing_decode_tokens":28383,"pending_prefill_tokens":20458,"num_requests":4,"active_p_offloads":0,"cached_blocks":1452,"cache_hit":0,"new_prefill":26146,"score_linear":120521.0,"score_lmetric":186416},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":26146,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":15283,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":26146,"score_linear":100467.0,"score_lmetric":98756},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":133694,"ongoing_decode_tokens":64229,"pending_prefill_tokens":3417,"num_requests":3,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":26146,"score_linear":133694.0,"score_lmetric":88689},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":54922,"ongoing_decode_tokens":54922,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":26146,"score_linear":54922.0,"score_lmetric":52292},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":47401,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":20992,"new_prefill":5154,"score_linear":26409.0,"score_lmetric":5154},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227886,"ongoing_decode_tokens":204169,"pending_prefill_tokens":165,"num_requests":3,"active_p_offloads":0,"cached_blocks":1405,"cache_hit":0,"new_prefill":26146,"score_linear":227886.0,"score_lmetric":78933}],"chosen_score_linear":26409.0,"chosen_score_lmetric":5154,"t_first_token":282066.213717309,"t_first_token_unix":1779712931.227661,"t_done":282066.712155917,"t_done_unix":1779712931.7260993},{"request_id":"1342921:3:1384020:670","session_id":"1342921","input_length":69465,"t_proxy_recv":282059.869062011,"t_decision_unix":1779712924.8830032,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":66048,"affinity_cache_ratio":0.950809760310948,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":66048,"estimated_new_tokens":3417,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":69465,"score_linear":148472.0,"score_lmetric":405154},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28383,"ongoing_decode_tokens":0,"pending_prefill_tokens":14559,"num_requests":1,"active_p_offloads":0,"cached_blocks":1452,"cache_hit":0,"new_prefill":69465,"score_linear":28383.0,"score_lmetric":84024},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1818,"cache_hit":0,"new_prefill":69465,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76740,"ongoing_decode_tokens":76740,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":69465,"score_linear":76740.0,"score_lmetric":138930},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":64229,"ongoing_decode_tokens":0,"pending_prefill_tokens":56549,"num_requests":2,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":66048,"new_prefill":3417,"score_linear":-1819.0,"score_lmetric":119932},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1772,"cache_hit":0,"new_prefill":69465,"score_linear":99424.0,"score_lmetric":138930},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":47401,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":69465,"score_linear":47401.0,"score_lmetric":69465},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":69465,"score_linear":227626.0,"score_lmetric":346104}],"chosen_score_linear":-1819.0,"chosen_score_lmetric":119932,"t_first_token":282065.618849796,"t_first_token_unix":1779712930.6327932,"t_done":282067.329915348,"t_done_unix":1779712932.3438587},{"request_id":"1383625:2:1384279:673","session_id":"1383625","input_length":4150,"t_proxy_recv":282060.911486504,"t_decision_unix":1779712925.9254277,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":3072,"affinity_cache_ratio":0.7402409638554217,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":3072,"estimated_new_tokens":1078,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":4150,"score_linear":148472.0,"score_lmetric":274524},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99637,"ongoing_decode_tokens":0,"pending_prefill_tokens":26421,"num_requests":2,"active_p_offloads":0,"cached_blocks":1452,"cache_hit":3072,"new_prefill":1078,"score_linear":96565.0,"score_lmetric":54998},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1818,"cache_hit":0,"new_prefill":4150,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76740,"ongoing_decode_tokens":76740,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":4150,"score_linear":76740.0,"score_lmetric":8300},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":133694,"ongoing_decode_tokens":0,"pending_prefill_tokens":59966,"num_requests":3,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":4150,"score_linear":133694.0,"score_lmetric":192348},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1772,"cache_hit":0,"new_prefill":4150,"score_linear":99424.0,"score_lmetric":8300},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":68622,"ongoing_decode_tokens":47401,"pending_prefill_tokens":229,"num_requests":2,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":4150,"score_linear":68622.0,"score_lmetric":8758},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":4150,"score_linear":227626.0,"score_lmetric":150159}],"chosen_score_linear":96565.0,"chosen_score_lmetric":54998,"t_first_token":282066.166301782,"t_first_token_unix":1779712931.1802447,"t_done":282067.399777105,"t_done_unix":1779712932.4137204},{"request_id":"1382086:1:1382086:654","session_id":"1382086","input_length":54012,"t_proxy_recv":282053.183726714,"t_decision_unix":1779712918.197668,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":54012,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":54012,"score_linear":148472.0,"score_lmetric":374248},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":13989,"ongoing_decode_tokens":13989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1377,"cache_hit":0,"new_prefill":54012,"score_linear":13989.0,"score_lmetric":54012},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38111,"ongoing_decode_tokens":0,"pending_prefill_tokens":38111,"num_requests":1,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":54012,"score_linear":38111.0,"score_lmetric":92123},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":54012,"score_linear":134396.0,"score_lmetric":179919},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":54012,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":54012,"score_linear":99424.0,"score_lmetric":108024},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":0,"pending_prefill_tokens":19753,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":54012,"score_linear":47401.0,"score_lmetric":73765},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":54012,"score_linear":227626.0,"score_lmetric":299745}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282064.128249896,"t_first_token_unix":1779712929.142193,"t_done":282067.593467206,"t_done_unix":1779712932.6074102},{"request_id":"1364090:3:1381959:653","session_id":"1364090","input_length":47401,"t_proxy_recv":282052.688777956,"t_decision_unix":1779712917.7027192,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":27648,"affinity_cache_ratio":0.5832788337798781,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":27648,"estimated_new_tokens":19753,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":7680,"new_prefill":39721,"score_linear":140792.0,"score_lmetric":345666},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":73363,"ongoing_decode_tokens":73363,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":4096,"new_prefill":43305,"score_linear":69267.0,"score_lmetric":86610},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":47401,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134396,"ongoing_decode_tokens":15283,"pending_prefill_tokens":5961,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":7680,"new_prefill":39721,"score_linear":126716.0,"score_lmetric":137046},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":47401,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":47401,"score_linear":99424.0,"score_lmetric":94802},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":27648,"new_prefill":19753,"score_linear":-27648.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":7680,"new_prefill":39721,"score_linear":219946.0,"score_lmetric":256872}],"chosen_score_linear":-27648.0,"chosen_score_lmetric":0,"t_first_token":282057.814909869,"t_first_token_unix":1779712922.8288531,"t_done":282067.615725469,"t_done_unix":1779712932.6296687},{"request_id":"1383090:2:1385262:682","session_id":"1383090","input_length":16734,"t_proxy_recv":282064.194582747,"t_decision_unix":1779712929.208524,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":9216,"affinity_cache_ratio":0.5507350304768734,"affinity_num_requests":3,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"cache_hit":9216,"estimated_new_tokens":7518,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":0,"pending_prefill_tokens":133313,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":16734,"score_linear":198337.0,"score_lmetric":450141},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103787,"ongoing_decode_tokens":28383,"pending_prefill_tokens":12940,"num_requests":3,"active_p_offloads":0,"cached_blocks":1452,"cache_hit":9216,"new_prefill":7518,"score_linear":94571.0,"score_lmetric":61374},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":16734,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":15283,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":16734,"score_linear":100467.0,"score_lmetric":79932},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":133694,"ongoing_decode_tokens":64229,"pending_prefill_tokens":3417,"num_requests":3,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":16734,"score_linear":133694.0,"score_lmetric":60453},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":54922,"ongoing_decode_tokens":35888,"pending_prefill_tokens":19034,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":16734,"score_linear":54922.0,"score_lmetric":71536},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":68622,"ongoing_decode_tokens":68622,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":16734,"score_linear":68622.0,"score_lmetric":33468},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":227626,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":16734,"score_linear":227626.0,"score_lmetric":50202}],"chosen_score_linear":94571.0,"chosen_score_lmetric":61374,"t_first_token":282066.992662461,"t_first_token_unix":1779712932.0066059,"t_done":282067.664954153,"t_done_unix":1779712932.6788976},{"request_id":"1355484:9:1379722:640","session_id":"1355484","input_length":23717,"t_proxy_recv":282065.243711485,"t_decision_unix":1779712930.257653,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":23552,"affinity_cache_ratio":0.9930429649618417,"affinity_num_requests":2,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":23552,"estimated_new_tokens":165,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":0,"pending_prefill_tokens":133313,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":23717,"score_linear":198337.0,"score_lmetric":471090},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120521,"ongoing_decode_tokens":28383,"pending_prefill_tokens":20458,"num_requests":4,"active_p_offloads":0,"cached_blocks":1452,"cache_hit":0,"new_prefill":23717,"score_linear":120521.0,"score_lmetric":176700},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":23717,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":15283,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":23717,"score_linear":100467.0,"score_lmetric":93898},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":133694,"ongoing_decode_tokens":64229,"pending_prefill_tokens":3417,"num_requests":3,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":23717,"score_linear":133694.0,"score_lmetric":81402},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":54922,"ongoing_decode_tokens":54922,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":23717,"score_linear":54922.0,"score_lmetric":47434},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":47401,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":23717,"score_linear":47401.0,"score_lmetric":23717},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":204169,"ongoing_decode_tokens":204169,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1405,"cache_hit":23552,"new_prefill":165,"score_linear":180617.0,"score_lmetric":330}],"chosen_score_linear":180617.0,"chosen_score_lmetric":330,"t_first_token":282065.354515545,"t_first_token_unix":1779712930.3684587,"t_done":282067.791622689,"t_done_unix":1779712932.8055665},{"request_id":"1313181:5:1320817:349","session_id":"1313181","input_length":105902,"t_proxy_recv":282037.648382444,"t_decision_unix":1779712902.662324,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":60416,"affinity_cache_ratio":0.5704896980227002,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":60416,"estimated_new_tokens":45486,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":105902,"score_linear":126340.0,"score_lmetric":232242},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":20727,"ongoing_decode_tokens":20727,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1368,"cache_hit":0,"new_prefill":105902,"score_linear":20727.0,"score_lmetric":105902},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":105902,"score_linear":79495.0,"score_lmetric":185397},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":56286,"ongoing_decode_tokens":56286,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":105902,"score_linear":56286.0,"score_lmetric":105902},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":135109,"ongoing_decode_tokens":16057,"pending_prefill_tokens":780,"num_requests":2,"active_p_offloads":0,"cached_blocks":1545,"cache_hit":0,"new_prefill":105902,"score_linear":135109.0,"score_lmetric":213364},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160794,"ongoing_decode_tokens":124906,"pending_prefill_tokens":48,"num_requests":4,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":105902,"score_linear":160794.0,"score_lmetric":423800},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":105902,"score_linear":80246.0,"score_lmetric":105902},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":60416,"new_prefill":45486,"score_linear":37851.0,"score_lmetric":45486}],"chosen_score_linear":37851.0,"chosen_score_lmetric":45486,"t_first_token":282061.032007392,"t_first_token_unix":1779712926.0459538,"t_done":282068.216068257,"t_done_unix":1779712933.2300117},{"request_id":"1340290:5:1386504:688","session_id":"1340290","input_length":27220,"t_proxy_recv":282068.448972306,"t_decision_unix":1779712933.4629135,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":26112,"affinity_cache_ratio":0.9592946362968405,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":26112,"estimated_new_tokens":1108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":0,"pending_prefill_tokens":133313,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":27220,"score_linear":198337.0,"score_lmetric":481599},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99637,"ongoing_decode_tokens":99637,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1468,"cache_hit":0,"new_prefill":27220,"score_linear":99637.0,"score_lmetric":54440},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":27220,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":15283,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":27220,"score_linear":100467.0,"score_lmetric":100904},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":27220,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":27220,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1507,"cache_hit":26112,"new_prefill":1108,"score_linear":-26112.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":184351,"ongoing_decode_tokens":184351,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1493,"cache_hit":0,"new_prefill":27220,"score_linear":184351.0,"score_lmetric":81660}],"chosen_score_linear":-26112.0,"chosen_score_lmetric":0,"t_first_token":282068.735171328,"t_first_token_unix":1779712933.7491152,"t_done":282068.943740041,"t_done_unix":1779712933.9576836},{"request_id":"1366128:11:1384044:671","session_id":"1366128","input_length":71254,"t_proxy_recv":282059.984262341,"t_decision_unix":1779712924.998204,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":59392,"affinity_cache_ratio":0.8335251354309934,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":59392,"estimated_new_tokens":11862,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":71254,"score_linear":148472.0,"score_lmetric":408732},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28383,"ongoing_decode_tokens":0,"pending_prefill_tokens":14559,"num_requests":1,"active_p_offloads":0,"cached_blocks":1452,"cache_hit":59392,"new_prefill":11862,"score_linear":-31009.0,"score_lmetric":26421},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1818,"cache_hit":0,"new_prefill":71254,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76740,"ongoing_decode_tokens":76740,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":71254,"score_linear":76740.0,"score_lmetric":142508},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":133694,"ongoing_decode_tokens":0,"pending_prefill_tokens":59966,"num_requests":3,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":71254,"score_linear":133694.0,"score_lmetric":393660},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1772,"cache_hit":0,"new_prefill":71254,"score_linear":99424.0,"score_lmetric":142508},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":47401,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":71254,"score_linear":47401.0,"score_lmetric":71254},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":71254,"score_linear":227626.0,"score_lmetric":351471}],"chosen_score_linear":-31009.0,"chosen_score_lmetric":26421,"t_first_token":282066.165471359,"t_first_token_unix":1779712931.1794147,"t_done":282069.034981329,"t_done_unix":1779712934.0489244},{"request_id":"1366128:12:1386045:687","session_id":"1366128","input_length":71578,"t_proxy_recv":282069.049216431,"t_decision_unix":1779712934.0631578,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9942719830115399,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":71168,"estimated_new_tokens":410,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":0,"pending_prefill_tokens":133313,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":71578,"score_linear":198337.0,"score_lmetric":614673},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28383,"ongoing_decode_tokens":28383,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1491,"cache_hit":71168,"new_prefill":410,"score_linear":-42785.0,"score_lmetric":410},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":71578,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":15283,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":71578,"score_linear":100467.0,"score_lmetric":189620},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":71578,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":71578,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":71578,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":184351,"ongoing_decode_tokens":184351,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1493,"cache_hit":0,"new_prefill":71578,"score_linear":184351.0,"score_lmetric":214734}],"chosen_score_linear":-42785.0,"chosen_score_lmetric":410,"t_first_token":282069.287268988,"t_first_token_unix":1779712934.301212,"t_done":282069.889354999,"t_done_unix":1779712934.9032984},{"request_id":"1381387:2:1383735:669","session_id":"1381387","input_length":28383,"t_proxy_recv":282058.805433541,"t_decision_unix":1779712923.8193743,"policy":"unified","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":13824,"affinity_cache_ratio":0.4870521086565902,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":14559,"tie_break_used":false,"cache_hit":13824,"estimated_new_tokens":14559,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":28383,"score_linear":148472.0,"score_lmetric":322990},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":3186,"ongoing_decode_tokens":3186,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1446,"cache_hit":13824,"new_prefill":14559,"score_linear":-10638.0,"score_lmetric":14559},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38111,"ongoing_decode_tokens":0,"pending_prefill_tokens":38111,"num_requests":1,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":28383,"score_linear":38111.0,"score_lmetric":66494},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76740,"ongoing_decode_tokens":76740,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":28383,"score_linear":76740.0,"score_lmetric":56766},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":64229,"ongoing_decode_tokens":0,"pending_prefill_tokens":56549,"num_requests":2,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":28383,"score_linear":64229.0,"score_lmetric":169864},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1772,"cache_hit":0,"new_prefill":28383,"score_linear":99424.0,"score_lmetric":56766},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47401,"ongoing_decode_tokens":47401,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":28383,"score_linear":47401.0,"score_lmetric":28383},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":28383,"score_linear":227626.0,"score_lmetric":222858}],"chosen_score_linear":-10638.0,"chosen_score_lmetric":14559,"t_first_token":282061.188117541,"t_first_token_unix":1779712926.202061,"t_done":282070.17242909,"t_done_unix":1779712935.1863723},{"request_id":"1381728:2:1387040:690","session_id":"1381728","input_length":12399,"t_proxy_recv":282070.227963034,"t_decision_unix":1779712935.241904,"policy":"unified","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":1024,"affinity_cache_ratio":0.08258730542785708,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":1024,"estimated_new_tokens":11375,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":0,"pending_prefill_tokens":133313,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":12399,"score_linear":198337.0,"score_lmetric":437136},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":12399,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":12399,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":15283,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":12399,"score_linear":100467.0,"score_lmetric":71262},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":12399,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":12399,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":1024,"new_prefill":11375,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":184351,"ongoing_decode_tokens":184351,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1493,"cache_hit":0,"new_prefill":12399,"score_linear":184351.0,"score_lmetric":37197}],"chosen_score_linear":-1024.0,"chosen_score_lmetric":0,"t_first_token":282071.179728638,"t_first_token_unix":1779712936.193672,"t_done":282071.265711822,"t_done_unix":1779712936.2796547},{"request_id":"1384772:2:1387335:693","session_id":"1384772","input_length":11530,"t_proxy_recv":282071.326380834,"t_decision_unix":1779712936.3403223,"policy":"unified","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":4608,"affinity_cache_ratio":0.39965307892454466,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":6922,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":0,"pending_prefill_tokens":133313,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":11530,"score_linear":198337.0,"score_lmetric":434529},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":122501,"ongoing_decode_tokens":0,"pending_prefill_tokens":23173,"num_requests":2,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":11530,"score_linear":122501.0,"score_lmetric":69406},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":4608,"new_prefill":6922,"score_linear":-4608.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":15283,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":11530,"score_linear":100467.0,"score_lmetric":69524},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":11530,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":11530,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":0,"new_prefill":11530,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":184351,"ongoing_decode_tokens":184351,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1493,"cache_hit":0,"new_prefill":11530,"score_linear":184351.0,"score_lmetric":34590}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":282071.974566138,"t_first_token_unix":1779712936.9885094,"t_done":282072.033470776,"t_done_unix":1779712937.0474136},{"request_id":"1355484:10:1384484:675","session_id":"1355484","input_length":25226,"t_proxy_recv":282067.798290069,"t_decision_unix":1779712932.8122315,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":23552,"affinity_cache_ratio":0.9336398953460715,"affinity_num_requests":2,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":23552,"estimated_new_tokens":1674,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":0,"pending_prefill_tokens":133313,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":25226,"score_linear":198337.0,"score_lmetric":475617},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99637,"ongoing_decode_tokens":99637,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1468,"cache_hit":0,"new_prefill":25226,"score_linear":99637.0,"score_lmetric":50452},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":25226,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":15283,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":25226,"score_linear":100467.0,"score_lmetric":96916},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":25226,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":25226,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1507,"cache_hit":0,"new_prefill":25226,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":204169,"ongoing_decode_tokens":204169,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1405,"cache_hit":23552,"new_prefill":1674,"score_linear":180617.0,"score_lmetric":3348}],"chosen_score_linear":180617.0,"chosen_score_lmetric":3348,"t_first_token":282068.155568146,"t_first_token_unix":1779712933.1695113,"t_done":282072.137044561,"t_done_unix":1779712937.1509879},{"request_id":"1279412:25:1366203:554","session_id":"1279412","input_length":98267,"t_proxy_recv":281997.536454377,"t_decision_unix":1779712862.5503957,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":97792,"affinity_cache_ratio":0.9951662307794071,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":97792,"estimated_new_tokens":475,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1055,"cache_hit":0,"new_prefill":98267,"score_linear":68259.0,"score_lmetric":98267},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":11154,"ongoing_decode_tokens":0,"pending_prefill_tokens":11154,"num_requests":1,"active_p_offloads":0,"cached_blocks":1093,"cache_hit":0,"new_prefill":98267,"score_linear":11154.0,"score_lmetric":109421},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150402,"ongoing_decode_tokens":0,"pending_prefill_tokens":122754,"num_requests":2,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":98267,"score_linear":150402.0,"score_lmetric":442042},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":91690,"ongoing_decode_tokens":0,"pending_prefill_tokens":78378,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":98267,"score_linear":91690.0,"score_lmetric":353290},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":98267,"score_linear":199977.0,"score_lmetric":656652},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":98267,"score_linear":62685.0,"score_lmetric":160952},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":0,"new_prefill":98267,"score_linear":80246.0,"score_lmetric":98267},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":59514,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":1,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":97792,"new_prefill":475,"score_linear":-38278.0,"score_lmetric":35925}],"chosen_score_linear":-38278.0,"chosen_score_lmetric":35925,"t_first_token":282006.351183937,"t_first_token_unix":1779712871.3651273,"t_done":282072.325652249,"t_done_unix":1779712937.3395956},{"request_id":"1340290:6:1387365:695","session_id":"1340290","input_length":27285,"t_proxy_recv":282071.430126445,"t_decision_unix":1779712936.4440675,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9945391240608393,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":27136,"estimated_new_tokens":149,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":0,"pending_prefill_tokens":133313,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":27285,"score_linear":198337.0,"score_lmetric":481794},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":122501,"ongoing_decode_tokens":0,"pending_prefill_tokens":23173,"num_requests":2,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":27285,"score_linear":122501.0,"score_lmetric":100916},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":11530,"ongoing_decode_tokens":0,"pending_prefill_tokens":6922,"num_requests":1,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":27285,"score_linear":11530.0,"score_lmetric":34207},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":15283,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":27285,"score_linear":100467.0,"score_lmetric":101034},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":27285,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":27285,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":27136,"new_prefill":149,"score_linear":-27136.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":184351,"ongoing_decode_tokens":184351,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1493,"cache_hit":0,"new_prefill":27285,"score_linear":184351.0,"score_lmetric":81855}],"chosen_score_linear":-27136.0,"chosen_score_lmetric":0,"t_first_token":282071.488836964,"t_first_token_unix":1779712936.50278,"t_done":282072.611926439,"t_done_unix":1779712937.6258695},{"request_id":"1279412:26:1378269:628","session_id":"1279412","input_length":100340,"t_proxy_recv":282072.344971692,"t_decision_unix":1779712937.358913,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":99840,"affinity_cache_ratio":0.9950169423958541,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":99840,"estimated_new_tokens":500,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":0,"pending_prefill_tokens":133313,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":100340,"score_linear":198337.0,"score_lmetric":700959},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":122501,"ongoing_decode_tokens":0,"pending_prefill_tokens":23173,"num_requests":2,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":100340,"score_linear":122501.0,"score_lmetric":247026},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":100340,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":15283,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":100340,"score_linear":100467.0,"score_lmetric":247144},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":100340,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":100340,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":27285,"ongoing_decode_tokens":27285,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":0,"new_prefill":100340,"score_linear":27285.0,"score_lmetric":100340},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60858,"ongoing_decode_tokens":60858,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":99840,"new_prefill":500,"score_linear":-38982.0,"score_lmetric":500}],"chosen_score_linear":-38982.0,"chosen_score_lmetric":500,"t_first_token":282072.624457945,"t_first_token_unix":1779712937.6384013,"t_done":282074.565909934,"t_done_unix":1779712939.5798533},{"request_id":"1382804:2:1388620:698","session_id":"1382804","input_length":3678,"t_proxy_recv":282075.936385962,"t_decision_unix":1779712940.9503272,"policy":"unified","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":512,"affinity_cache_ratio":0.13920609026644915,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":512,"estimated_new_tokens":3166,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":126340,"pending_prefill_tokens":6973,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":3678,"score_linear":198337.0,"score_lmetric":31953},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":122501,"ongoing_decode_tokens":0,"pending_prefill_tokens":23173,"num_requests":2,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":3678,"score_linear":122501.0,"score_lmetric":53702},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":3678,"score_linear":71899.0,"score_lmetric":75577},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":100467,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":3678,"score_linear":100467.0,"score_lmetric":7356},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":3678,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":512,"new_prefill":3166,"score_linear":-512.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":0,"new_prefill":3678,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":188233,"ongoing_decode_tokens":188233,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1498,"cache_hit":0,"new_prefill":3678,"score_linear":188233.0,"score_lmetric":11034}],"chosen_score_linear":-512.0,"chosen_score_lmetric":0,"t_first_token":282076.138159719,"t_first_token_unix":1779712941.1521032,"t_done":282076.191163218,"t_done_unix":1779712941.2051063},{"request_id":"1366128:13:1387308:692","session_id":"1366128","input_length":71708,"t_proxy_recv":282071.251809145,"t_decision_unix":1779712936.265751,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9924694594745356,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":71168,"estimated_new_tokens":540,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":0,"pending_prefill_tokens":133313,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":71708,"score_linear":198337.0,"score_lmetric":615063},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":50793,"ongoing_decode_tokens":0,"pending_prefill_tokens":22633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":71168,"new_prefill":540,"score_linear":-20375.0,"score_lmetric":23173},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":71708,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":15283,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":71708,"score_linear":100467.0,"score_lmetric":189880},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":71708,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":71708,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12399,"ongoing_decode_tokens":12399,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":71708,"score_linear":12399.0,"score_lmetric":71708},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":184351,"ongoing_decode_tokens":184351,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1493,"cache_hit":0,"new_prefill":71708,"score_linear":184351.0,"score_lmetric":215124}],"chosen_score_linear":-20375.0,"chosen_score_lmetric":23173,"t_first_token":282076.561212913,"t_first_token_unix":1779712941.5751562,"t_done":282077.036395003,"t_done_unix":1779712942.0503387},{"request_id":"1355484:11:1388276:696","session_id":"1355484","input_length":26837,"t_proxy_recv":282074.560594792,"t_decision_unix":1779712939.574536,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":25088,"affinity_cache_ratio":0.9348287811603383,"affinity_num_requests":2,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":25088,"estimated_new_tokens":1749,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":126340,"pending_prefill_tokens":6973,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":26837,"score_linear":198337.0,"score_lmetric":101430},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":122501,"ongoing_decode_tokens":0,"pending_prefill_tokens":23173,"num_requests":2,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":26837,"score_linear":122501.0,"score_lmetric":100020},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":26837,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":100467,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":26837,"score_linear":100467.0,"score_lmetric":53674},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":26837,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":26837,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":0,"new_prefill":26837,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":161198,"ongoing_decode_tokens":161198,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":25088,"new_prefill":1749,"score_linear":136110.0,"score_lmetric":3498}],"chosen_score_linear":136110.0,"chosen_score_lmetric":3498,"t_first_token":282074.920693624,"t_first_token_unix":1779712939.9346375,"t_done":282077.827205933,"t_done_unix":1779712942.8411493},{"request_id":"1384947:2:1388842:700","session_id":"1384947","input_length":20183,"t_proxy_recv":282076.696735844,"t_decision_unix":1779712941.7106771,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":18944,"affinity_cache_ratio":0.9386117029182975,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":18944,"estimated_new_tokens":1239,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":198337,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":20183,"score_linear":198337.0,"score_lmetric":60549},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":122501,"ongoing_decode_tokens":122501,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":20183,"score_linear":122501.0,"score_lmetric":40366},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":20183,"score_linear":71899.0,"score_lmetric":92082},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":100467,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":20183,"score_linear":100467.0,"score_lmetric":40366},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":20183,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1818,"cache_hit":18944,"new_prefill":1239,"score_linear":-18944.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":0,"new_prefill":20183,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":188233,"ongoing_decode_tokens":188233,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1498,"cache_hit":0,"new_prefill":20183,"score_linear":188233.0,"score_lmetric":60549}],"chosen_score_linear":-18944.0,"chosen_score_lmetric":0,"t_first_token":282076.934836461,"t_first_token_unix":1779712941.9487805,"t_done":282078.00634712,"t_done_unix":1779712943.0202901},{"request_id":"1279412:27:1380696:646","session_id":"1279412","input_length":100538,"t_proxy_recv":282074.583908283,"t_decision_unix":1779712939.5978498,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":100352,"affinity_cache_ratio":0.9981499532515069,"affinity_num_requests":2,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":100352,"estimated_new_tokens":186,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":126340,"pending_prefill_tokens":6973,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":100538,"score_linear":198337.0,"score_lmetric":322533},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":122501,"ongoing_decode_tokens":0,"pending_prefill_tokens":23173,"num_requests":2,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":100538,"score_linear":122501.0,"score_lmetric":247422},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":100538,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":100467,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":100538,"score_linear":100467.0,"score_lmetric":201076},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":100538,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":100538,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":0,"new_prefill":100538,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":87695,"ongoing_decode_tokens":60858,"pending_prefill_tokens":1749,"num_requests":2,"active_p_offloads":0,"cached_blocks":1498,"cache_hit":100352,"new_prefill":186,"score_linear":-12657.0,"score_lmetric":3870}],"chosen_score_linear":-12657.0,"chosen_score_lmetric":3870,"t_first_token":282075.074256884,"t_first_token_unix":1779712940.0882008,"t_done":282078.284089572,"t_done_unix":1779712943.2980332},{"request_id":"1340278:3:1384549:676","session_id":"1340278","input_length":49865,"t_proxy_recv":282061.727033597,"t_decision_unix":1779712926.740975,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9959691166148601,"affinity_num_requests":2,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":201,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":49664,"new_prefill":201,"score_linear":98808.0,"score_lmetric":266626},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103787,"ongoing_decode_tokens":28383,"pending_prefill_tokens":12940,"num_requests":3,"active_p_offloads":0,"cached_blocks":1452,"cache_hit":0,"new_prefill":49865,"score_linear":103787.0,"score_lmetric":188415},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1829,"cache_hit":0,"new_prefill":49865,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76740,"ongoing_decode_tokens":76740,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":49865,"score_linear":76740.0,"score_lmetric":99730},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":133694,"ongoing_decode_tokens":0,"pending_prefill_tokens":59966,"num_requests":3,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":49865,"score_linear":133694.0,"score_lmetric":329493},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":99424,"ongoing_decode_tokens":99424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1772,"cache_hit":0,"new_prefill":49865,"score_linear":99424.0,"score_lmetric":99730},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":68622,"ongoing_decode_tokens":47401,"pending_prefill_tokens":229,"num_requests":2,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":49865,"score_linear":68622.0,"score_lmetric":100188},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":227626,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":48640,"new_prefill":1225,"score_linear":178986.0,"score_lmetric":3675}],"chosen_score_linear":98808.0,"chosen_score_lmetric":266626,"t_first_token":282076.628368056,"t_first_token_unix":1779712941.642311,"t_done":282079.040913462,"t_done_unix":1779712944.054857},{"request_id":"1340278:3:1385417:683","session_id":"1340278","input_length":50272,"t_proxy_recv":282079.06949038,"t_decision_unix":1779712944.0834317,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9879057924888606,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":608,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":148472,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":49664,"new_prefill":608,"score_linear":98808.0,"score_lmetric":1216},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":134197,"ongoing_decode_tokens":50793,"pending_prefill_tokens":11724,"num_requests":2,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":50272,"score_linear":134197.0,"score_lmetric":123992},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":50272,"score_linear":71899.0,"score_lmetric":122171},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":100467,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":50272,"score_linear":100467.0,"score_lmetric":100544},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":70990,"ongoing_decode_tokens":0,"pending_prefill_tokens":1870,"num_requests":1,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":50272,"score_linear":70990.0,"score_lmetric":52142},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1820,"cache_hit":0,"new_prefill":50272,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":0,"new_prefill":50272,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":161624,"ongoing_decode_tokens":161624,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":48640,"new_prefill":1632,"score_linear":112984.0,"score_lmetric":3264}],"chosen_score_linear":98808.0,"chosen_score_lmetric":1216,"t_first_token":282079.331075333,"t_first_token_unix":1779712944.3450189,"t_done":282079.370662975,"t_done_unix":1779712944.3846064},{"request_id":"1313181:5:1374956:615","session_id":"1313181","input_length":60858,"t_proxy_recv":282068.2282215,"t_decision_unix":1779712933.2421627,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":60416,"affinity_cache_ratio":0.9927371914949554,"affinity_num_requests":2,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":60416,"estimated_new_tokens":442,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":0,"pending_prefill_tokens":133313,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":60858,"score_linear":198337.0,"score_lmetric":582513},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99637,"ongoing_decode_tokens":99637,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1468,"cache_hit":0,"new_prefill":60858,"score_linear":99637.0,"score_lmetric":121716},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":60858,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":15283,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":60858,"score_linear":100467.0,"score_lmetric":168180},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":60858,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":60858,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1507,"cache_hit":0,"new_prefill":60858,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123493,"ongoing_decode_tokens":123493,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1493,"cache_hit":60416,"new_prefill":442,"score_linear":63077.0,"score_lmetric":884}],"chosen_score_linear":63077.0,"chosen_score_lmetric":884,"t_first_token":282068.349476206,"t_first_token_unix":1779712933.3634195,"t_done":282079.395393451,"t_done_unix":1779712944.4093368},{"request_id":"1370129:4:1384856:678","session_id":"1370129","input_length":85184,"t_proxy_recv":282063.797519874,"t_decision_unix":1779712928.8114612,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":61952,"affinity_cache_ratio":0.7272727272727273,"affinity_num_requests":1,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"cache_hit":61952,"estimated_new_tokens":23232,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":0,"pending_prefill_tokens":133313,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":7680,"new_prefill":77504,"score_linear":190657.0,"score_lmetric":632451},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103787,"ongoing_decode_tokens":28383,"pending_prefill_tokens":12940,"num_requests":3,"active_p_offloads":0,"cached_blocks":1452,"cache_hit":4096,"new_prefill":81088,"score_linear":99691.0,"score_lmetric":282084},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":11113,"ongoing_decode_tokens":0,"pending_prefill_tokens":11113,"num_requests":1,"active_p_offloads":0,"cached_blocks":1838,"cache_hit":0,"new_prefill":85184,"score_linear":11113.0,"score_lmetric":96297},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":61952,"new_prefill":23232,"score_linear":-46669.0,"score_lmetric":23232},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":133694,"ongoing_decode_tokens":0,"pending_prefill_tokens":59966,"num_requests":3,"active_p_offloads":0,"cached_blocks":1583,"cache_hit":0,"new_prefill":85184,"score_linear":133694.0,"score_lmetric":435450},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":54922,"ongoing_decode_tokens":35888,"pending_prefill_tokens":19034,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":85184,"score_linear":54922.0,"score_lmetric":208436},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":68622,"ongoing_decode_tokens":68622,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":7680,"new_prefill":77504,"score_linear":60942.0,"score_lmetric":155008},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":227626,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":7680,"new_prefill":77504,"score_linear":219946.0,"score_lmetric":232512}],"chosen_score_linear":-46669.0,"chosen_score_lmetric":23232,"t_first_token":282074.48716767,"t_first_token_unix":1779712939.5011113,"t_done":282081.162865555,"t_done_unix":1779712946.176809},{"request_id":"1286804:4:1378156:627","session_id":"1286804","input_length":22132,"t_proxy_recv":282039.377392167,"t_decision_unix":1779712904.3913336,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":15360,"affinity_cache_ratio":0.694017711910356,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":15360,"estimated_new_tokens":6772,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":15360,"new_prefill":6772,"score_linear":110980.0,"score_lmetric":133112},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":59007,"ongoing_decode_tokens":59007,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1371,"cache_hit":0,"new_prefill":22132,"score_linear":59007.0,"score_lmetric":22132},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":22132,"score_linear":79495.0,"score_lmetric":101627},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71137,"ongoing_decode_tokens":71137,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":22132,"score_linear":71137.0,"score_lmetric":44264},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":119052,"ongoing_decode_tokens":119052,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1550,"cache_hit":0,"new_prefill":22132,"score_linear":119052.0,"score_lmetric":22132},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160794,"ongoing_decode_tokens":124906,"pending_prefill_tokens":48,"num_requests":4,"active_p_offloads":0,"cached_blocks":1714,"cache_hit":0,"new_prefill":22132,"score_linear":160794.0,"score_lmetric":88720},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":22132,"score_linear":80246.0,"score_lmetric":22132},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":22132,"score_linear":227626.0,"score_lmetric":204105}],"chosen_score_linear":110980.0,"chosen_score_lmetric":133112,"t_first_token":282076.626459807,"t_first_token_unix":1779712941.640403,"t_done":282081.50685319,"t_done_unix":1779712946.5207968},{"request_id":"1342921:4:1389290:702","session_id":"1342921","input_length":70990,"t_proxy_recv":282078.352394968,"t_decision_unix":1779712943.3663366,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":69120,"affinity_cache_ratio":0.9736582617270038,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":69120,"estimated_new_tokens":1870,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":198337,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":70990,"score_linear":198337.0,"score_lmetric":212970},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":134197,"ongoing_decode_tokens":50793,"pending_prefill_tokens":11724,"num_requests":2,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":70990,"score_linear":134197.0,"score_lmetric":165428},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":70990,"score_linear":71899.0,"score_lmetric":142889},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":100467,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":70990,"score_linear":100467.0,"score_lmetric":141980},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":69120,"new_prefill":1870,"score_linear":-69120.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1820,"cache_hit":0,"new_prefill":70990,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":0,"new_prefill":70990,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":161624,"ongoing_decode_tokens":60858,"pending_prefill_tokens":414,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":70990,"score_linear":161624.0,"score_lmetric":142808}],"chosen_score_linear":-69120.0,"chosen_score_lmetric":0,"t_first_token":282079.124334243,"t_first_token_unix":1779712944.1382775,"t_done":282081.546344552,"t_done_unix":1779712946.560288},{"request_id":"1390240:1:1390240:705","session_id":"1390240","input_length":5110,"t_proxy_recv":282081.872892207,"t_decision_unix":1779712946.8868334,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5110,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":126340,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":5110,"score_linear":126340.0,"score_lmetric":5110},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":134197,"ongoing_decode_tokens":50793,"pending_prefill_tokens":11724,"num_requests":2,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":5110,"score_linear":134197.0,"score_lmetric":33668},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":5110,"score_linear":71899.0,"score_lmetric":77009},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":0,"new_prefill":5110,"score_linear":15283.0,"score_lmetric":5110},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1703,"cache_hit":0,"new_prefill":5110,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68187,"ongoing_decode_tokens":0,"pending_prefill_tokens":3163,"num_requests":1,"active_p_offloads":0,"cached_blocks":1820,"cache_hit":0,"new_prefill":5110,"score_linear":68187.0,"score_lmetric":8273},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":0,"new_prefill":5110,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":5110,"score_linear":245313.0,"score_lmetric":150987}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282082.197707155,"t_first_token_unix":1779712947.2116506,"t_done":282082.47170791,"t_done_unix":1779712947.4856513},{"request_id":"1390356:1:1390356:706","session_id":"1390356","input_length":1349,"t_proxy_recv":282082.313485171,"t_decision_unix":1779712947.3274262,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":1349,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":126340,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":1349,"score_linear":126340.0,"score_lmetric":1349},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":134197,"ongoing_decode_tokens":50793,"pending_prefill_tokens":11724,"num_requests":2,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":1349,"score_linear":134197.0,"score_lmetric":26146},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":1349,"score_linear":71899.0,"score_lmetric":73248},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":0,"new_prefill":1349,"score_linear":15283.0,"score_lmetric":1349},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1703,"cache_hit":0,"new_prefill":1349,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68187,"ongoing_decode_tokens":0,"pending_prefill_tokens":3163,"num_requests":1,"active_p_offloads":0,"cached_blocks":1820,"cache_hit":0,"new_prefill":1349,"score_linear":68187.0,"score_lmetric":4512},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":5110,"ongoing_decode_tokens":5110,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":0,"new_prefill":1349,"score_linear":5110.0,"score_lmetric":1349},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":1349,"score_linear":245313.0,"score_lmetric":139704}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282082.409041111,"t_first_token_unix":1779712947.4229853,"t_done":282084.100322403,"t_done_unix":1779712949.1142657},{"request_id":"1373431:1:1373431:606","session_id":"1373431","input_length":126340,"t_proxy_recv":282023.04199285,"t_decision_unix":1779712888.0559342,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":126340,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":126340,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1290,"cache_hit":0,"new_prefill":126340,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":126340,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":126340,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199977,"ongoing_decode_tokens":0,"pending_prefill_tokens":120617,"num_requests":3,"active_p_offloads":0,"cached_blocks":1313,"cache_hit":0,"new_prefill":126340,"score_linear":199977.0,"score_lmetric":740871},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151655,"ongoing_decode_tokens":151655,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":1024,"new_prefill":125316,"score_linear":150631.0,"score_lmetric":375948},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1349,"cache_hit":0,"new_prefill":126340,"score_linear":106951.0,"score_lmetric":252680},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181263,"ongoing_decode_tokens":181263,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":126340,"score_linear":181263.0,"score_lmetric":379020}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282074.43482202,"t_first_token_unix":1779712939.448766,"t_done":282084.220258586,"t_done_unix":1779712949.234202},{"request_id":"1381387:3:1387093:691","session_id":"1381387","input_length":50793,"t_proxy_recv":282070.438915317,"t_decision_unix":1779712935.4528565,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":28160,"affinity_cache_ratio":0.5544071033410115,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":28160,"estimated_new_tokens":22633,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":0,"pending_prefill_tokens":133313,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":50793,"score_linear":198337.0,"score_lmetric":552318},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":28160,"new_prefill":22633,"score_linear":-28160.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":50793,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":15283,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":50793,"score_linear":100467.0,"score_lmetric":148050},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":50793,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":50793,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12399,"ongoing_decode_tokens":0,"pending_prefill_tokens":11375,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":50793,"score_linear":12399.0,"score_lmetric":62168},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":184351,"ongoing_decode_tokens":184351,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1493,"cache_hit":0,"new_prefill":50793,"score_linear":184351.0,"score_lmetric":152379}],"chosen_score_linear":-28160.0,"chosen_score_lmetric":0,"t_first_token":282076.556465649,"t_first_token_unix":1779712941.5704093,"t_done":282085.504996665,"t_done_unix":1779712950.5189402},{"request_id":"1366128:14:1389156:701","session_id":"1366128","input_length":83404,"t_proxy_recv":282077.92552938,"t_decision_unix":1779712942.9394705,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":71680,"affinity_cache_ratio":0.8594312023404154,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":71680,"estimated_new_tokens":11724,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":198337,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":83404,"score_linear":198337.0,"score_lmetric":250212},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":50793,"ongoing_decode_tokens":50793,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":71680,"new_prefill":11724,"score_linear":-20887.0,"score_lmetric":11724},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":83404,"score_linear":71899.0,"score_lmetric":155303},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":100467,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":83404,"score_linear":100467.0,"score_lmetric":166808},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":83404,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":20183,"ongoing_decode_tokens":20183,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1818,"cache_hit":0,"new_prefill":83404,"score_linear":20183.0,"score_lmetric":83404},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":0,"new_prefill":83404,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":161396,"ongoing_decode_tokens":161396,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":83404,"score_linear":161396.0,"score_lmetric":166808}],"chosen_score_linear":-20887.0,"chosen_score_lmetric":11724,"t_first_token":282083.772251765,"t_first_token_unix":1779712948.786199,"t_done":282085.54085035,"t_done_unix":1779712950.5547938},{"request_id":"1373431:2:1378320:629","session_id":"1373431","input_length":127267,"t_proxy_recv":282084.238852383,"t_decision_unix":1779712949.252794,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":126464,"affinity_cache_ratio":0.9936904303550803,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":126464,"estimated_new_tokens":803,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1494,"cache_hit":126464,"new_prefill":803,"score_linear":-126464.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":157838,"ongoing_decode_tokens":157838,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":127267,"score_linear":157838.0,"score_lmetric":381801},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":127267,"score_linear":71899.0,"score_lmetric":199166},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":102970,"ongoing_decode_tokens":102970,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":0,"new_prefill":127267,"score_linear":102970.0,"score_lmetric":254534},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1706,"cache_hit":1024,"new_prefill":126243,"score_linear":-1024.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68187,"ongoing_decode_tokens":68187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1820,"cache_hit":1024,"new_prefill":126243,"score_linear":67163.0,"score_lmetric":126243},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":127267,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":127267,"score_linear":245313.0,"score_lmetric":517458}],"chosen_score_linear":-126464.0,"chosen_score_lmetric":0,"t_first_token":282084.859801393,"t_first_token_unix":1779712949.873745,"t_done":282086.479528278,"t_done_unix":1779712951.4934719},{"request_id":"1363093:4:1390050:704","session_id":"1363093","input_length":68187,"t_proxy_recv":282081.116181716,"t_decision_unix":1779712946.130123,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9536128587560678,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":65024,"estimated_new_tokens":3163,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":148472,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1234,"cache_hit":0,"new_prefill":68187,"score_linear":148472.0,"score_lmetric":136374},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":134197,"ongoing_decode_tokens":50793,"pending_prefill_tokens":11724,"num_requests":2,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":68187,"score_linear":134197.0,"score_lmetric":159822},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":68187,"score_linear":71899.0,"score_lmetric":140086},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":100467,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":68187,"score_linear":100467.0,"score_lmetric":136374},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":70990,"ongoing_decode_tokens":70990,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":68187,"score_linear":70990.0,"score_lmetric":68187},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1820,"cache_hit":65024,"new_prefill":3163,"score_linear":-65024.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":0,"new_prefill":68187,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":68187,"score_linear":245313.0,"score_lmetric":340218}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":282082.368629116,"t_first_token_unix":1779712947.3825734,"t_done":282086.58245606,"t_done_unix":1779712951.5963995},{"request_id":"1384947:3:1390898:710","session_id":"1384947","input_length":20683,"t_proxy_recv":282084.524988858,"t_decision_unix":1779712949.5389302,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":19968,"affinity_cache_ratio":0.9654305468258957,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":715,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":127267,"ongoing_decode_tokens":0,"pending_prefill_tokens":803,"num_requests":1,"active_p_offloads":0,"cached_blocks":1494,"cache_hit":0,"new_prefill":20683,"score_linear":127267.0,"score_lmetric":21486},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":157838,"ongoing_decode_tokens":157838,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":20683,"score_linear":157838.0,"score_lmetric":62049},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":20683,"score_linear":71899.0,"score_lmetric":92582},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":102970,"ongoing_decode_tokens":102970,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":0,"new_prefill":20683,"score_linear":102970.0,"score_lmetric":41366},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1706,"cache_hit":0,"new_prefill":20683,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68187,"ongoing_decode_tokens":68187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1820,"cache_hit":19968,"new_prefill":715,"score_linear":48219.0,"score_lmetric":715},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":20683,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":20683,"score_linear":245313.0,"score_lmetric":197706}],"chosen_score_linear":48219.0,"chosen_score_lmetric":715,"t_first_token":282084.664707244,"t_first_token_unix":1779712949.6786509,"t_done":282086.640021536,"t_done_unix":1779712951.6539671},{"request_id":"1366128:15:1390663:709","session_id":"1366128","input_length":83622,"t_proxy_recv":282085.556060077,"t_decision_unix":1779712950.5700016,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":83456,"affinity_cache_ratio":0.9980148764679151,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":83456,"estimated_new_tokens":166,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":127267,"ongoing_decode_tokens":127267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1494,"cache_hit":0,"new_prefill":83622,"score_linear":127267.0,"score_lmetric":83622},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23641,"ongoing_decode_tokens":23641,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1587,"cache_hit":83456,"new_prefill":166,"score_linear":-59815.0,"score_lmetric":166},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":83622,"score_linear":71899.0,"score_lmetric":155521},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":102970,"ongoing_decode_tokens":102970,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":0,"new_prefill":83622,"score_linear":102970.0,"score_lmetric":167244},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1706,"cache_hit":0,"new_prefill":83622,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":88870,"ongoing_decode_tokens":88870,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1820,"cache_hit":0,"new_prefill":83622,"score_linear":88870.0,"score_lmetric":167244},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":83622,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":83622,"score_linear":245313.0,"score_lmetric":386523}],"chosen_score_linear":-59815.0,"chosen_score_lmetric":166,"t_first_token":282085.772675308,"t_first_token_unix":1779712950.7866187,"t_done":282087.106465264,"t_done_unix":1779712952.1204085},{"request_id":"1391584:1:1391584:713","session_id":"1391584","input_length":15302,"t_proxy_recv":282086.387334596,"t_decision_unix":1779712951.4012759,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":15302,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":127267,"ongoing_decode_tokens":127267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1494,"cache_hit":0,"new_prefill":15302,"score_linear":127267.0,"score_lmetric":15302},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":107263,"ongoing_decode_tokens":107263,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1587,"cache_hit":0,"new_prefill":15302,"score_linear":107263.0,"score_lmetric":30604},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":15302,"score_linear":71899.0,"score_lmetric":87201},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":102970,"ongoing_decode_tokens":102970,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":0,"new_prefill":15302,"score_linear":102970.0,"score_lmetric":30604},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":28307,"ongoing_decode_tokens":0,"pending_prefill_tokens":28307,"num_requests":1,"active_p_offloads":0,"cached_blocks":1706,"cache_hit":0,"new_prefill":15302,"score_linear":28307.0,"score_lmetric":43609},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":88870,"ongoing_decode_tokens":88870,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1820,"cache_hit":0,"new_prefill":15302,"score_linear":88870.0,"score_lmetric":30604},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":15302,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":15302,"score_linear":245313.0,"score_lmetric":181563}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282087.758174625,"t_first_token_unix":1779712952.7721183,"t_done":282087.972390404,"t_done_unix":1779712952.9863334},{"request_id":"1373431:3:1383388:667","session_id":"1373431","input_length":128201,"t_proxy_recv":282086.499820778,"t_decision_unix":1779712951.513762,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":126976,"affinity_cache_ratio":0.9904446923190927,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":126976,"estimated_new_tokens":1225,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1495,"cache_hit":126976,"new_prefill":1225,"score_linear":-126976.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":107263,"ongoing_decode_tokens":107263,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1587,"cache_hit":0,"new_prefill":128201,"score_linear":107263.0,"score_lmetric":256402},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":128201,"score_linear":71899.0,"score_lmetric":200100},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":102970,"ongoing_decode_tokens":102970,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":0,"new_prefill":128201,"score_linear":102970.0,"score_lmetric":256402},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":28307,"ongoing_decode_tokens":0,"pending_prefill_tokens":28307,"num_requests":1,"active_p_offloads":0,"cached_blocks":1706,"cache_hit":1024,"new_prefill":127177,"score_linear":27283.0,"score_lmetric":155484},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":88870,"ongoing_decode_tokens":88870,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1820,"cache_hit":1024,"new_prefill":127177,"score_linear":87846.0,"score_lmetric":254354},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":15302,"ongoing_decode_tokens":0,"pending_prefill_tokens":15302,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":128201,"score_linear":15302.0,"score_lmetric":143503},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":128201,"score_linear":245313.0,"score_lmetric":520260}],"chosen_score_linear":-126976.0,"chosen_score_lmetric":0,"t_first_token":282087.361733603,"t_first_token_unix":1779712952.375678,"t_done":282088.369740409,"t_done_unix":1779712953.3836842},{"request_id":"1370129:5:1390477:708","session_id":"1370129","input_length":87687,"t_proxy_recv":282082.845490278,"t_decision_unix":1779712947.8594317,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":85504,"affinity_cache_ratio":0.9751046335260642,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":85504,"estimated_new_tokens":2183,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":126340,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":7680,"new_prefill":80007,"score_linear":118660.0,"score_lmetric":80007},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":157838,"ongoing_decode_tokens":50793,"pending_prefill_tokens":13861,"num_requests":3,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":4096,"new_prefill":83591,"score_linear":153742.0,"score_lmetric":292356},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":87687,"score_linear":71899.0,"score_lmetric":159586},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":85504,"new_prefill":2183,"score_linear":-70221.0,"score_lmetric":2183},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":1349,"ongoing_decode_tokens":1349,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1703,"cache_hit":0,"new_prefill":87687,"score_linear":1349.0,"score_lmetric":87687},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68187,"ongoing_decode_tokens":68187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1820,"cache_hit":0,"new_prefill":87687,"score_linear":68187.0,"score_lmetric":87687},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":7680,"new_prefill":80007,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":7680,"new_prefill":80007,"score_linear":237633.0,"score_lmetric":375678}],"chosen_score_linear":-70221.0,"chosen_score_lmetric":2183,"t_first_token":282084.101964545,"t_first_token_unix":1779712949.115908,"t_done":282089.261871329,"t_done_unix":1779712954.2758198},{"request_id":"1391584:2:1392466:715","session_id":"1391584","input_length":15483,"t_proxy_recv":282089.710298674,"t_decision_unix":1779712954.7242396,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":14848,"affinity_cache_ratio":0.958987276367629,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":14848,"estimated_new_tokens":635,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":15483,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23641,"ongoing_decode_tokens":23641,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1587,"cache_hit":0,"new_prefill":15483,"score_linear":23641.0,"score_lmetric":15483},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":15483,"score_linear":71899.0,"score_lmetric":87382},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":15483,"score_linear":15283.0,"score_lmetric":15483},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":28307,"ongoing_decode_tokens":28307,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1706,"cache_hit":0,"new_prefill":15483,"score_linear":28307.0,"score_lmetric":15483},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":28382,"ongoing_decode_tokens":0,"pending_prefill_tokens":28382,"num_requests":1,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":15483,"score_linear":28382.0,"score_lmetric":43865},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1570,"cache_hit":14848,"new_prefill":635,"score_linear":-14848.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":15483,"score_linear":245313.0,"score_lmetric":182106}],"chosen_score_linear":-14848.0,"chosen_score_lmetric":0,"t_first_token":282089.784761544,"t_first_token_unix":1779712954.7987053,"t_done":282090.171647606,"t_done_unix":1779712955.1855907},{"request_id":"1363943:9:1390468:707","session_id":"1363943","input_length":23641,"t_proxy_recv":282082.789762038,"t_decision_unix":1779712947.8037033,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":21504,"affinity_cache_ratio":0.9096061926314454,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":21504,"estimated_new_tokens":2137,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126340,"ongoing_decode_tokens":126340,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":8704,"new_prefill":14937,"score_linear":117636.0,"score_lmetric":14937},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":134197,"ongoing_decode_tokens":50793,"pending_prefill_tokens":11724,"num_requests":2,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":21504,"new_prefill":2137,"score_linear":112693.0,"score_lmetric":27722},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":23641,"score_linear":71899.0,"score_lmetric":95540},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":4096,"new_prefill":19545,"score_linear":11187.0,"score_lmetric":19545},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":1349,"ongoing_decode_tokens":1349,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1703,"cache_hit":0,"new_prefill":23641,"score_linear":1349.0,"score_lmetric":23641},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68187,"ongoing_decode_tokens":68187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1820,"cache_hit":0,"new_prefill":23641,"score_linear":68187.0,"score_lmetric":23641},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":4096,"new_prefill":19545,"score_linear":-4096.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":4096,"new_prefill":19545,"score_linear":241217.0,"score_lmetric":194292}],"chosen_score_linear":112693.0,"chosen_score_lmetric":27722,"t_first_token":282084.046367633,"t_first_token_unix":1779712949.0603118,"t_done":282090.386090808,"t_done_unix":1779712955.4000342},{"request_id":"1391404:1:1391404:712","session_id":"1391404","input_length":28307,"t_proxy_recv":282085.872367723,"t_decision_unix":1779712950.886309,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":28307,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":127267,"ongoing_decode_tokens":127267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1494,"cache_hit":2560,"new_prefill":25747,"score_linear":124707.0,"score_lmetric":25747},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":107263,"ongoing_decode_tokens":107263,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1587,"cache_hit":0,"new_prefill":28307,"score_linear":107263.0,"score_lmetric":56614},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":28307,"score_linear":71899.0,"score_lmetric":100206},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":102970,"ongoing_decode_tokens":102970,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":0,"new_prefill":28307,"score_linear":102970.0,"score_lmetric":56614},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1706,"cache_hit":0,"new_prefill":28307,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":88870,"ongoing_decode_tokens":88870,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1820,"cache_hit":0,"new_prefill":28307,"score_linear":88870.0,"score_lmetric":56614},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":28307,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":28307,"score_linear":245313.0,"score_lmetric":220578}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282089.438693186,"t_first_token_unix":1779712954.452637,"t_done":282090.763485908,"t_done_unix":1779712955.777429},{"request_id":"1390240:2:1392954:717","session_id":"1390240","input_length":7440,"t_proxy_recv":282091.421053745,"t_decision_unix":1779712956.434995,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":5120,"affinity_cache_ratio":0.6881720430107527,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":5120,"estimated_new_tokens":2320,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":7440,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":84897,"ongoing_decode_tokens":84897,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":7440,"score_linear":84897.0,"score_lmetric":7440},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":7440,"score_linear":71899.0,"score_lmetric":79339},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":7440,"score_linear":15283.0,"score_lmetric":7440},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":7440,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":28382,"ongoing_decode_tokens":28382,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":7440,"score_linear":28382.0,"score_lmetric":7440},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1571,"cache_hit":5120,"new_prefill":2320,"score_linear":-5120.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":7440,"score_linear":245313.0,"score_lmetric":157977}],"chosen_score_linear":-5120.0,"chosen_score_lmetric":0,"t_first_token":282091.646896115,"t_first_token_unix":1779712956.6608398,"t_done":282092.302428504,"t_done_unix":1779712957.3163717},{"request_id":"1355484:12:1391879:714","session_id":"1355484","input_length":28382,"t_proxy_recv":282087.525126989,"t_decision_unix":1779712952.5390682,"policy":"unified","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":5,"affinity_cache_hit":26624,"affinity_cache_ratio":0.9380593333803114,"affinity_num_requests":3,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":28382,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":128201,"ongoing_decode_tokens":128201,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1495,"cache_hit":0,"new_prefill":28382,"score_linear":128201.0,"score_lmetric":28382},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23641,"ongoing_decode_tokens":23641,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1587,"cache_hit":0,"new_prefill":28382,"score_linear":23641.0,"score_lmetric":28382},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":28382,"score_linear":71899.0,"score_lmetric":100281},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":102970,"ongoing_decode_tokens":102970,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":0,"new_prefill":28382,"score_linear":102970.0,"score_lmetric":56764},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":28307,"ongoing_decode_tokens":0,"pending_prefill_tokens":28307,"num_requests":1,"active_p_offloads":0,"cached_blocks":1706,"cache_hit":0,"new_prefill":28382,"score_linear":28307.0,"score_lmetric":56689},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":28382,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":15302,"ongoing_decode_tokens":0,"pending_prefill_tokens":15302,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":28382,"score_linear":15302.0,"score_lmetric":43684},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":26624,"new_prefill":1758,"score_linear":218689.0,"score_lmetric":140931}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282091.118402921,"t_first_token_unix":1779712956.132347,"t_done":282092.753015984,"t_done_unix":1779712957.7669597},{"request_id":"1366128:16:1392704:716","session_id":"1366128","input_length":84897,"t_proxy_recv":282090.605792013,"t_decision_unix":1779712955.6197333,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":83456,"affinity_cache_ratio":0.9830264909242965,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":83456,"estimated_new_tokens":1441,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":84897,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":83456,"new_prefill":1441,"score_linear":-83456.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":84897,"score_linear":71899.0,"score_lmetric":156796},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":84897,"score_linear":15283.0,"score_lmetric":84897},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":28307,"ongoing_decode_tokens":28307,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1706,"cache_hit":0,"new_prefill":84897,"score_linear":28307.0,"score_lmetric":84897},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":28382,"ongoing_decode_tokens":0,"pending_prefill_tokens":28382,"num_requests":1,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":84897,"score_linear":28382.0,"score_lmetric":113279},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1571,"cache_hit":0,"new_prefill":84897,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":84897,"score_linear":245313.0,"score_lmetric":390348}],"chosen_score_linear":-83456.0,"chosen_score_lmetric":0,"t_first_token":282091.355679604,"t_first_token_unix":1779712956.3696237,"t_done":282093.287644959,"t_done_unix":1779712958.3015883},{"request_id":"1388507:1:1388507:697","session_id":"1388507","input_length":71899,"t_proxy_recv":282075.479545588,"t_decision_unix":1779712940.493487,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":71899,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":126340,"pending_prefill_tokens":6973,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":71899,"score_linear":198337.0,"score_lmetric":236616},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":122501,"ongoing_decode_tokens":0,"pending_prefill_tokens":23173,"num_requests":2,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":71899,"score_linear":122501.0,"score_lmetric":190144},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":71899,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":100467,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":71899,"score_linear":100467.0,"score_lmetric":143798},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":71899,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":71899,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":0,"new_prefill":71899,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":188233,"ongoing_decode_tokens":188233,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1498,"cache_hit":0,"new_prefill":71899,"score_linear":188233.0,"score_lmetric":215697}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282093.322807296,"t_first_token_unix":1779712958.3367507,"t_done":282093.673345721,"t_done_unix":1779712958.6872888},{"request_id":"1394167:1:1394167:721","session_id":"1394167","input_length":4675,"t_proxy_recv":282095.685503343,"t_decision_unix":1779712960.6994445,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4675,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":4675,"score_linear":101803.0,"score_lmetric":33262},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1595,"cache_hit":0,"new_prefill":4675,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2012,"cache_hit":0,"new_prefill":4675,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":4675,"score_linear":15283.0,"score_lmetric":4675},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":0,"pending_prefill_tokens":5161,"num_requests":1,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":4675,"score_linear":33321.0,"score_lmetric":9836},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":0,"new_prefill":4675,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":48944,"ongoing_decode_tokens":48944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1575,"cache_hit":0,"new_prefill":4675,"score_linear":48944.0,"score_lmetric":4675},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":4675,"score_linear":245313.0,"score_lmetric":149682}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282095.97782762,"t_first_token_unix":1779712960.991771,"t_done":282096.354688575,"t_done_unix":1779712961.3686318},{"request_id":"1394428:1:1394428:724","session_id":"1394428","input_length":982,"t_proxy_recv":282096.572911669,"t_decision_unix":1779712961.5868528,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":982,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":512,"new_prefill":470,"score_linear":101291.0,"score_lmetric":29057},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1595,"cache_hit":0,"new_prefill":982,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2021,"cache_hit":0,"new_prefill":982,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121426,"ongoing_decode_tokens":15283,"pending_prefill_tokens":18079,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":982,"score_linear":121426.0,"score_lmetric":38122},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":982,"score_linear":33321.0,"score_lmetric":982},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":28665,"ongoing_decode_tokens":28665,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":0,"new_prefill":982,"score_linear":28665.0,"score_lmetric":982},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":48944,"ongoing_decode_tokens":48944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1575,"cache_hit":0,"new_prefill":982,"score_linear":48944.0,"score_lmetric":982},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":982,"score_linear":245313.0,"score_lmetric":138603}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282096.650432873,"t_first_token_unix":1779712961.6643763,"t_done":282096.653952292,"t_done_unix":1779712961.6678956},{"request_id":"1364090:4:1392998:718","session_id":"1364090","input_length":48944,"t_proxy_recv":282091.52653761,"t_decision_unix":1779712956.540479,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":48128,"affinity_cache_ratio":0.9833278849297156,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":48128,"estimated_new_tokens":816,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":7680,"new_prefill":41264,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":84897,"ongoing_decode_tokens":84897,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":4096,"new_prefill":44848,"score_linear":80801.0,"score_lmetric":44848},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":48944,"score_linear":71899.0,"score_lmetric":120843},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":7680,"new_prefill":41264,"score_linear":7603.0,"score_lmetric":41264},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":48944,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":28382,"ongoing_decode_tokens":28382,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":48944,"score_linear":28382.0,"score_lmetric":48944},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7440,"ongoing_decode_tokens":0,"pending_prefill_tokens":2320,"num_requests":1,"active_p_offloads":0,"cached_blocks":1571,"cache_hit":48128,"new_prefill":816,"score_linear":-40688.0,"score_lmetric":3136},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":7680,"new_prefill":41264,"score_linear":237633.0,"score_lmetric":259449}],"chosen_score_linear":-40688.0,"chosen_score_lmetric":3136,"t_first_token":282091.978729771,"t_first_token_unix":1779712956.9926748,"t_done":282096.712248804,"t_done_unix":1779712961.7261922},{"request_id":"1355484:13:1394384:723","session_id":"1355484","input_length":28665,"t_proxy_recv":282096.430167231,"t_decision_unix":1779712961.4441085,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":28160,"affinity_cache_ratio":0.9823826966684109,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":28160,"estimated_new_tokens":505,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":28665,"score_linear":101803.0,"score_lmetric":57252},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1595,"cache_hit":0,"new_prefill":28665,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2021,"cache_hit":0,"new_prefill":28665,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121426,"ongoing_decode_tokens":15283,"pending_prefill_tokens":18079,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":28665,"score_linear":121426.0,"score_lmetric":93488},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":28665,"score_linear":33321.0,"score_lmetric":28665},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":28160,"new_prefill":505,"score_linear":-28160.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":48944,"ongoing_decode_tokens":48944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1575,"cache_hit":0,"new_prefill":28665,"score_linear":48944.0,"score_lmetric":28665},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":26624,"new_prefill":2041,"score_linear":218689.0,"score_lmetric":141780}],"chosen_score_linear":-28160.0,"chosen_score_lmetric":0,"t_first_token":282096.491591598,"t_first_token_unix":1779712961.5055346,"t_done":282096.935441447,"t_done_unix":1779712961.9493845},{"request_id":"1394432:1:1394432:725","session_id":"1394432","input_length":4736,"t_proxy_recv":282096.590454498,"t_decision_unix":1779712961.604396,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":4736,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":4736,"score_linear":101803.0,"score_lmetric":33323},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1595,"cache_hit":0,"new_prefill":4736,"score_linear":982.0,"score_lmetric":5718},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2021,"cache_hit":0,"new_prefill":4736,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121426,"ongoing_decode_tokens":15283,"pending_prefill_tokens":18079,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":4736,"score_linear":121426.0,"score_lmetric":45630},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":4736,"score_linear":33321.0,"score_lmetric":4736},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":28665,"ongoing_decode_tokens":28665,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":0,"new_prefill":4736,"score_linear":28665.0,"score_lmetric":4736},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":48944,"ongoing_decode_tokens":48944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1575,"cache_hit":0,"new_prefill":4736,"score_linear":48944.0,"score_lmetric":4736},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":4736,"score_linear":245313.0,"score_lmetric":149865}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282096.889149042,"t_first_token_unix":1779712961.9030921,"t_done":282097.149713517,"t_done_unix":1779712962.1636565},{"request_id":"1394482:1:1394482:726","session_id":"1394482","input_length":4428,"t_proxy_recv":282096.843601673,"t_decision_unix":1779712961.8575432,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4428,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":4428,"score_linear":101803.0,"score_lmetric":33015},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":4428,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":4736,"ongoing_decode_tokens":0,"pending_prefill_tokens":4736,"num_requests":1,"active_p_offloads":0,"cached_blocks":2021,"cache_hit":0,"new_prefill":4428,"score_linear":4736.0,"score_lmetric":9164},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121426,"ongoing_decode_tokens":15283,"pending_prefill_tokens":18079,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":4428,"score_linear":121426.0,"score_lmetric":45014},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":4428,"score_linear":33321.0,"score_lmetric":4428},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":28665,"ongoing_decode_tokens":28665,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":0,"new_prefill":4428,"score_linear":28665.0,"score_lmetric":4428},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1577,"cache_hit":0,"new_prefill":4428,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":4428,"score_linear":245313.0,"score_lmetric":148941}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282097.118694291,"t_first_token_unix":1779712962.132638,"t_done":282097.207078454,"t_done_unix":1779712962.2210214},{"request_id":"1394432:2:1395186:729","session_id":"1394432","input_length":6892,"t_proxy_recv":282099.178248785,"t_decision_unix":1779712964.1921904,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":4608,"affinity_cache_ratio":0.6686012768427162,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":2284,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":6892,"score_linear":101803.0,"score_lmetric":35479},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":6892,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":4863,"ongoing_decode_tokens":4863,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2030,"cache_hit":4608,"new_prefill":2284,"score_linear":255.0,"score_lmetric":2284},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121426,"ongoing_decode_tokens":15283,"pending_prefill_tokens":18079,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":6892,"score_linear":121426.0,"score_lmetric":49942},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":6892,"score_linear":33321.0,"score_lmetric":6892},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":35450,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":1,"active_p_offloads":0,"cached_blocks":1884,"cache_hit":0,"new_prefill":6892,"score_linear":35450.0,"score_lmetric":42342},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":6892,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":6892,"score_linear":245313.0,"score_lmetric":156333}],"chosen_score_linear":255.0,"chosen_score_lmetric":2284,"t_first_token":282099.382785918,"t_first_token_unix":1779712964.396729,"t_done":282099.676712683,"t_done_unix":1779712964.6906555},{"request_id":"1394167:2:1395165:728","session_id":"1394167","input_length":4863,"t_proxy_recv":282099.115555365,"t_decision_unix":1779712964.1294963,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":4608,"affinity_cache_ratio":0.9475632325724861,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":255,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":4863,"score_linear":101803.0,"score_lmetric":33450},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":4863,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2030,"cache_hit":4608,"new_prefill":255,"score_linear":-4608.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121426,"ongoing_decode_tokens":15283,"pending_prefill_tokens":18079,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":4863,"score_linear":121426.0,"score_lmetric":45884},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":4863,"score_linear":33321.0,"score_lmetric":4863},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":35450,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":1,"active_p_offloads":0,"cached_blocks":1884,"cache_hit":0,"new_prefill":4863,"score_linear":35450.0,"score_lmetric":40313},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":4863,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":4863,"score_linear":245313.0,"score_lmetric":150246}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":282099.153051465,"t_first_token_unix":1779712964.1669946,"t_done":282099.988207912,"t_done_unix":1779712965.002151},{"request_id":"1390240:3:1395387:730","session_id":"1390240","input_length":10890,"t_proxy_recv":282099.973086847,"t_decision_unix":1779712964.9870281,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":7168,"affinity_cache_ratio":0.65821854912764,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":7168,"estimated_new_tokens":3722,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":10890,"score_linear":101803.0,"score_lmetric":39477},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":10890,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":4863,"ongoing_decode_tokens":4863,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":0,"new_prefill":10890,"score_linear":4863.0,"score_lmetric":10890},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121426,"ongoing_decode_tokens":15283,"pending_prefill_tokens":18079,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":10890,"score_linear":121426.0,"score_lmetric":57938},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":10890,"score_linear":33321.0,"score_lmetric":10890},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":35450,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":1,"active_p_offloads":0,"cached_blocks":1884,"cache_hit":0,"new_prefill":10890,"score_linear":35450.0,"score_lmetric":46340},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":7168,"new_prefill":3722,"score_linear":-7168.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":10890,"score_linear":245313.0,"score_lmetric":168327}],"chosen_score_linear":-7168.0,"chosen_score_lmetric":0,"t_first_token":282100.334210015,"t_first_token_unix":1779712965.3481534,"t_done":282100.911261593,"t_done_unix":1779712965.9252048},{"request_id":"1366128:17:1395596:731","session_id":"1366128","input_length":85276,"t_proxy_recv":282100.734592228,"t_decision_unix":1779712965.7485337,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":84992,"affinity_cache_ratio":0.9966696374126366,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":84992,"estimated_new_tokens":284,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":85276,"score_linear":101803.0,"score_lmetric":113863},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":84992,"new_prefill":284,"score_linear":-84992.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":0,"new_prefill":85276,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121426,"ongoing_decode_tokens":15283,"pending_prefill_tokens":18079,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":85276,"score_linear":121426.0,"score_lmetric":206710},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":85276,"score_linear":33321.0,"score_lmetric":85276},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":35450,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":1,"active_p_offloads":0,"cached_blocks":1884,"cache_hit":0,"new_prefill":85276,"score_linear":35450.0,"score_lmetric":120726},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":10890,"ongoing_decode_tokens":10890,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":85276,"score_linear":10890.0,"score_lmetric":85276},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":85276,"score_linear":245313.0,"score_lmetric":391485}],"chosen_score_linear":-84992.0,"chosen_score_lmetric":0,"t_first_token":282101.005947746,"t_first_token_unix":1779712966.019891,"t_done":282101.914379212,"t_done_unix":1779712966.9283228},{"request_id":"1394696:1:1394696:727","session_id":"1394696","input_length":35450,"t_proxy_recv":282097.620546568,"t_decision_unix":1779712962.6344879,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":35450,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":35450,"score_linear":101803.0,"score_lmetric":64037},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":35450,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2030,"cache_hit":0,"new_prefill":35450,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121426,"ongoing_decode_tokens":15283,"pending_prefill_tokens":18079,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":35450,"score_linear":121426.0,"score_lmetric":107058},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":35450,"score_linear":33321.0,"score_lmetric":35450},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1884,"cache_hit":0,"new_prefill":35450,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":35450,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":35450,"score_linear":245313.0,"score_lmetric":242007}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282102.800137842,"t_first_token_unix":1779712967.8140812,"t_done":282103.217481565,"t_done_unix":1779712968.2314248},{"request_id":"1395933:1:1395933:732","session_id":"1395933","input_length":21241,"t_proxy_recv":282101.916145309,"t_decision_unix":1779712966.9300869,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":11776,"estimated_new_tokens":9465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":21241,"score_linear":101803.0,"score_lmetric":49828},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":11776,"new_prefill":9465,"score_linear":-11776.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":0,"new_prefill":21241,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121426,"ongoing_decode_tokens":15283,"pending_prefill_tokens":18079,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":21241,"score_linear":121426.0,"score_lmetric":78640},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":21241,"score_linear":33321.0,"score_lmetric":21241},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":35450,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":1,"active_p_offloads":0,"cached_blocks":1884,"cache_hit":0,"new_prefill":21241,"score_linear":35450.0,"score_lmetric":56691},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":11776,"new_prefill":9465,"score_linear":-11776.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":21241,"score_linear":245313.0,"score_lmetric":199380}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":0,"t_first_token":282104.171480524,"t_first_token_unix":1779712969.1854239,"t_done":282105.4636181,"t_done_unix":1779712970.4775612},{"request_id":"1355951:4:1389858:703","session_id":"1355951","input_length":38242,"t_proxy_recv":282080.477978653,"t_decision_unix":1779712945.4919193,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":37888,"affinity_cache_ratio":0.9907431619685163,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":37888,"estimated_new_tokens":354,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":148472,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1234,"cache_hit":0,"new_prefill":38242,"score_linear":148472.0,"score_lmetric":76484},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":134197,"ongoing_decode_tokens":50793,"pending_prefill_tokens":11724,"num_requests":2,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":38242,"score_linear":134197.0,"score_lmetric":99932},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":34816,"new_prefill":3426,"score_linear":37083.0,"score_lmetric":75325},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":100467,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":38242,"score_linear":100467.0,"score_lmetric":76484},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":70990,"ongoing_decode_tokens":70990,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":38242,"score_linear":70990.0,"score_lmetric":38242},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1820,"cache_hit":0,"new_prefill":38242,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":0,"new_prefill":38242,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":207071,"ongoing_decode_tokens":100766,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":37888,"new_prefill":354,"score_linear":169183.0,"score_lmetric":90438}],"chosen_score_linear":169183.0,"chosen_score_lmetric":90438,"t_first_token":282105.151695816,"t_first_token_unix":1779712970.1656394,"t_done":282106.027987692,"t_done_unix":1779712971.041931},{"request_id":"1378979:1:1378979:633","session_id":"1378979","input_length":15283,"t_proxy_recv":282042.2223455,"t_decision_unix":1779712907.2362869,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":14848,"estimated_new_tokens":435,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":0,"pending_prefill_tokens":133112,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":15283,"score_linear":148472.0,"score_lmetric":296790},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":15283,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":15283,"score_linear":79495.0,"score_lmetric":94778},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":14848,"new_prefill":435,"score_linear":-14848.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1566,"cache_hit":0,"new_prefill":15283,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":140932,"ongoing_decode_tokens":140932,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1752,"cache_hit":0,"new_prefill":15283,"score_linear":140932.0,"score_lmetric":45849},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1432,"cache_hit":0,"new_prefill":15283,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":227626,"ongoing_decode_tokens":98267,"pending_prefill_tokens":45903,"num_requests":3,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":15283,"score_linear":227626.0,"score_lmetric":183558}],"chosen_score_linear":-14848.0,"chosen_score_lmetric":0,"t_first_token":282043.583572539,"t_first_token_unix":1779712908.5975165,"t_done":282107.176678749,"t_done_unix":1779712972.190622},{"request_id":"1305906:3:1396717:733","session_id":"1305906","input_length":17047,"t_proxy_recv":282104.503959638,"t_decision_unix":1779712969.517901,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":16384,"affinity_cache_ratio":0.9611075262509532,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":16384,"estimated_new_tokens":663,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":17047,"score_linear":101803.0,"score_lmetric":45634},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":17047,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":0,"new_prefill":17047,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121426,"ongoing_decode_tokens":15283,"pending_prefill_tokens":18079,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":17047,"score_linear":121426.0,"score_lmetric":70252},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":16384,"new_prefill":663,"score_linear":16937.0,"score_lmetric":663},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":17047,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21241,"ongoing_decode_tokens":21241,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":17047,"score_linear":21241.0,"score_lmetric":17047},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":207071,"pending_prefill_tokens":354,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":17047,"score_linear":245313.0,"score_lmetric":52203}],"chosen_score_linear":16937.0,"chosen_score_lmetric":663,"t_first_token":282106.16480726,"t_first_token_unix":1779712971.1787508,"t_done":282107.30641252,"t_done_unix":1779712972.3203566},{"request_id":"1279412:28:1382526:657","session_id":"1279412","input_length":100766,"t_proxy_recv":282078.300901061,"t_decision_unix":1779712943.3148425,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":100352,"affinity_cache_ratio":0.9958914713296152,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":100352,"estimated_new_tokens":414,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198337,"ongoing_decode_tokens":198337,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":100766,"score_linear":198337.0,"score_lmetric":302298},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":134197,"ongoing_decode_tokens":50793,"pending_prefill_tokens":11724,"num_requests":2,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":100766,"score_linear":134197.0,"score_lmetric":224980},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":100766,"score_linear":71899.0,"score_lmetric":172665},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":100467,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":100766,"score_linear":100467.0,"score_lmetric":201532},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":100766,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1820,"cache_hit":0,"new_prefill":100766,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":0,"new_prefill":100766,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60858,"ongoing_decode_tokens":60858,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":100352,"new_prefill":414,"score_linear":-39494.0,"score_lmetric":414}],"chosen_score_linear":-39494.0,"chosen_score_lmetric":414,"t_first_token":282078.468171999,"t_first_token_unix":1779712943.4821155,"t_done":282107.37857326,"t_done_unix":1779712972.3925166},{"request_id":"1366128:18:1397382:735","session_id":"1366128","input_length":85504,"t_proxy_recv":282106.770313613,"t_decision_unix":1779712971.784255,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":84992,"affinity_cache_ratio":0.9940119760479041,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":84992,"estimated_new_tokens":512,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":85504,"score_linear":101803.0,"score_lmetric":114091},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":13134,"ongoing_decode_tokens":0,"pending_prefill_tokens":13134,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":84992,"new_prefill":512,"score_linear":-71858.0,"score_lmetric":13646},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":0,"new_prefill":85504,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121426,"ongoing_decode_tokens":121426,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":85504,"score_linear":121426.0,"score_lmetric":171008},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":50368,"ongoing_decode_tokens":50368,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":85504,"score_linear":50368.0,"score_lmetric":171008},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":85504,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":85504,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":207071,"ongoing_decode_tokens":207071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":85504,"score_linear":207071.0,"score_lmetric":171008}],"chosen_score_linear":-71858.0,"chosen_score_lmetric":13646,"t_first_token":282107.407084789,"t_first_token_unix":1779712972.4210281,"t_done":282108.308986673,"t_done_unix":1779712973.3229303},{"request_id":"1313181:6:1324908:369","session_id":"1313181","input_length":106305,"t_proxy_recv":282079.412775417,"t_decision_unix":1779712944.426717,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":61440,"affinity_cache_ratio":0.5779596444193594,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":61440,"estimated_new_tokens":44865,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":148472,"ongoing_decode_tokens":148472,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1234,"cache_hit":0,"new_prefill":106305,"score_linear":148472.0,"score_lmetric":212610},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":134197,"ongoing_decode_tokens":50793,"pending_prefill_tokens":11724,"num_requests":2,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":106305,"score_linear":134197.0,"score_lmetric":236058},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":106305,"score_linear":71899.0,"score_lmetric":178204},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100467,"ongoing_decode_tokens":100467,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":106305,"score_linear":100467.0,"score_lmetric":212610},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":70990,"ongoing_decode_tokens":70990,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1699,"cache_hit":0,"new_prefill":106305,"score_linear":70990.0,"score_lmetric":106305},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1820,"cache_hit":0,"new_prefill":106305,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":0,"new_prefill":106305,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":100766,"ongoing_decode_tokens":100766,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":61440,"new_prefill":44865,"score_linear":39326.0,"score_lmetric":44865}],"chosen_score_linear":39326.0,"chosen_score_lmetric":44865,"t_first_token":282103.581526206,"t_first_token_unix":1779712968.5954695,"t_done":282108.544742909,"t_done_unix":1779712973.5586908},{"request_id":"1279412:29:1385688:685","session_id":"1279412","input_length":101200,"t_proxy_recv":282107.397234928,"t_decision_unix":1779712972.411176,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":100864,"affinity_cache_ratio":0.9966798418972332,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":100864,"estimated_new_tokens":336,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":101200,"score_linear":101803.0,"score_lmetric":129787},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":98638,"ongoing_decode_tokens":13134,"pending_prefill_tokens":512,"num_requests":2,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":101200,"score_linear":98638.0,"score_lmetric":203424},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":0,"new_prefill":101200,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":101200,"score_linear":106143.0,"score_lmetric":101200},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":101200,"score_linear":33321.0,"score_lmetric":101200},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":93040,"num_requests":1,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":12800,"new_prefill":88400,"score_linear":80240.0,"score_lmetric":181440},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":101200,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106305,"ongoing_decode_tokens":106305,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1504,"cache_hit":100864,"new_prefill":336,"score_linear":5441.0,"score_lmetric":336}],"chosen_score_linear":5441.0,"chosen_score_lmetric":336,"t_first_token":282107.715417265,"t_first_token_unix":1779712972.7293627,"t_done":282108.830548152,"t_done_unix":1779712973.8444917},{"request_id":"1398145:1:1398145:739","session_id":"1398145","input_length":3213,"t_proxy_recv":282109.66993112,"t_decision_unix":1779712974.6838722,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":3213,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":3213,"score_linear":101803.0,"score_lmetric":31800},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":13134,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1597,"cache_hit":0,"new_prefill":3213,"score_linear":13134.0,"score_lmetric":3213},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16550,"ongoing_decode_tokens":0,"pending_prefill_tokens":16550,"num_requests":1,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":0,"new_prefill":3213,"score_linear":16550.0,"score_lmetric":19763},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":3213,"score_linear":106143.0,"score_lmetric":3213},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":3213,"score_linear":33321.0,"score_lmetric":3213},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":93040,"num_requests":1,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":3213,"score_linear":93040.0,"score_lmetric":96253},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":3213,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":162939,"ongoing_decode_tokens":162939,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":3213,"score_linear":162939.0,"score_lmetric":6426}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282109.867194033,"t_first_token_unix":1779712974.8811383,"t_done":282109.970511583,"t_done_unix":1779712974.9844546},{"request_id":"1362265:5:1398180:741","session_id":"1362265","input_length":10736,"t_proxy_recv":282109.754157877,"t_decision_unix":1779712974.7680995,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":10240,"affinity_cache_ratio":0.9538002980625931,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":10240,"estimated_new_tokens":496,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":6656,"new_prefill":4080,"score_linear":95147.0,"score_lmetric":32667},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":13134,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1597,"cache_hit":10240,"new_prefill":496,"score_linear":2894.0,"score_lmetric":496},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16550,"ongoing_decode_tokens":0,"pending_prefill_tokens":16550,"num_requests":1,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":0,"new_prefill":10736,"score_linear":16550.0,"score_lmetric":27286},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":6656,"new_prefill":4080,"score_linear":99487.0,"score_lmetric":4080},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":10736,"score_linear":33321.0,"score_lmetric":10736},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":10736,"score_linear":124264.0,"score_lmetric":212656},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":3213,"ongoing_decode_tokens":0,"pending_prefill_tokens":3213,"num_requests":1,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":10736,"score_linear":3213.0,"score_lmetric":13949},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":162939,"ongoing_decode_tokens":162939,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":10736,"score_linear":162939.0,"score_lmetric":21472}],"chosen_score_linear":2894.0,"chosen_score_lmetric":496,"t_first_token":282109.802584971,"t_first_token_unix":1779712974.8165283,"t_done":282110.032818344,"t_done_unix":1779712975.0467615},{"request_id":"1279412:30:1387344:694","session_id":"1279412","input_length":101396,"t_proxy_recv":282108.849348378,"t_decision_unix":1779712973.8632898,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":100864,"affinity_cache_ratio":0.9947532447039331,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":100864,"estimated_new_tokens":532,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":101396,"score_linear":101803.0,"score_lmetric":129983},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":13134,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1597,"cache_hit":0,"new_prefill":101396,"score_linear":13134.0,"score_lmetric":101396},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16550,"ongoing_decode_tokens":0,"pending_prefill_tokens":16550,"num_requests":1,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":0,"new_prefill":101396,"score_linear":16550.0,"score_lmetric":117946},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":101396,"score_linear":106143.0,"score_lmetric":101396},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":101396,"score_linear":33321.0,"score_lmetric":101396},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":93040,"num_requests":1,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":12800,"new_prefill":88596,"score_linear":80240.0,"score_lmetric":181636},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":101396,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":61543,"ongoing_decode_tokens":61543,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":100864,"new_prefill":532,"score_linear":-39321.0,"score_lmetric":532}],"chosen_score_linear":-39321.0,"chosen_score_lmetric":532,"t_first_token":282109.12987009,"t_first_token_unix":1779712974.143814,"t_done":282110.235068101,"t_done_unix":1779712975.249012},{"request_id":"1397840:1:1397840:737","session_id":"1397840","input_length":16550,"t_proxy_recv":282108.595334842,"t_decision_unix":1779712973.6092765,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":16550,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":16550,"score_linear":101803.0,"score_lmetric":45137},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":13134,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1597,"cache_hit":0,"new_prefill":16550,"score_linear":13134.0,"score_lmetric":16550},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":0,"new_prefill":16550,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":16550,"score_linear":106143.0,"score_lmetric":16550},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":16550,"score_linear":33321.0,"score_lmetric":16550},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":93040,"num_requests":1,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":16550,"score_linear":93040.0,"score_lmetric":109590},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":16550,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":162743,"ongoing_decode_tokens":101200,"pending_prefill_tokens":103,"num_requests":2,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":16550,"score_linear":162743.0,"score_lmetric":33306}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282110.153850748,"t_first_token_unix":1779712975.1677945,"t_done":282110.396890774,"t_done_unix":1779712975.410834},{"request_id":"1398213:1:1398213:742","session_id":"1398213","input_length":3891,"t_proxy_recv":282109.808938472,"t_decision_unix":1779712974.82288,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":3891,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3891,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":3891,"score_linear":101803.0,"score_lmetric":32478},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23870,"ongoing_decode_tokens":23870,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1597,"cache_hit":0,"new_prefill":3891,"score_linear":23870.0,"score_lmetric":7782},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16550,"ongoing_decode_tokens":0,"pending_prefill_tokens":16550,"num_requests":1,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":0,"new_prefill":3891,"score_linear":16550.0,"score_lmetric":20441},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":3891,"score_linear":106143.0,"score_lmetric":3891},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":3891,"score_linear":33321.0,"score_lmetric":3891},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":3891,"score_linear":124264.0,"score_lmetric":198966},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":3213,"ongoing_decode_tokens":0,"pending_prefill_tokens":3213,"num_requests":1,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":3891,"score_linear":3213.0,"score_lmetric":7104},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":162939,"ongoing_decode_tokens":162939,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":3891,"score_linear":162939.0,"score_lmetric":7782}],"chosen_score_linear":33321.0,"chosen_score_lmetric":3891,"t_first_token":282110.072409489,"t_first_token_unix":1779712975.0863528,"t_done":282110.566118769,"t_done_unix":1779712975.5800622},{"request_id":"1397223:1:1397223:734","session_id":"1397223","input_length":13134,"t_proxy_recv":282106.144061735,"t_decision_unix":1779712971.158003,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":13134,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":13134,"score_linear":101803.0,"score_lmetric":41721},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":13134,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":0,"new_prefill":13134,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":121426,"ongoing_decode_tokens":15283,"pending_prefill_tokens":18079,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":13134,"score_linear":121426.0,"score_lmetric":62426},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":50368,"ongoing_decode_tokens":33321,"pending_prefill_tokens":663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":13134,"score_linear":50368.0,"score_lmetric":27594},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":13134,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":13134,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":207071,"ongoing_decode_tokens":207071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":13134,"score_linear":207071.0,"score_lmetric":26268}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282107.26026257,"t_first_token_unix":1779712972.2742057,"t_done":282110.9170479,"t_done_unix":1779712975.9309914},{"request_id":"1398594:1:1398594:744","session_id":"1398594","input_length":7877,"t_proxy_recv":282111.037094567,"t_decision_unix":1779712976.051036,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7877,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":7877,"score_linear":101803.0,"score_lmetric":36464},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":0,"new_prefill":7877,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2066,"cache_hit":0,"new_prefill":7877,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":169401,"ongoing_decode_tokens":106143,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":7877,"score_linear":169401.0,"score_lmetric":39870},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1769,"cache_hit":0,"new_prefill":7877,"score_linear":33321.0,"score_lmetric":7877},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":7877,"score_linear":124264.0,"score_lmetric":206938},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":7877,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":163536,"ongoing_decode_tokens":163536,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1507,"cache_hit":0,"new_prefill":7877,"score_linear":163536.0,"score_lmetric":15754}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282111.579055152,"t_first_token_unix":1779712976.5929985,"t_done":282111.696423512,"t_done_unix":1779712976.7103667},{"request_id":"1366128:19:1398824:745","session_id":"1366128","input_length":85663,"t_proxy_recv":282111.93494885,"t_decision_unix":1779712976.948887,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":85504,"affinity_cache_ratio":0.9981438894271739,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":85504,"estimated_new_tokens":159,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":85663,"score_linear":101803.0,"score_lmetric":114250},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":85504,"new_prefill":159,"score_linear":-85504.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2081,"cache_hit":0,"new_prefill":85663,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":169401,"ongoing_decode_tokens":106143,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":85663,"score_linear":169401.0,"score_lmetric":195442},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1769,"cache_hit":0,"new_prefill":85663,"score_linear":33321.0,"score_lmetric":85663},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":85663,"score_linear":124264.0,"score_lmetric":362510},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":85663,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":163536,"ongoing_decode_tokens":163536,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1507,"cache_hit":0,"new_prefill":85663,"score_linear":163536.0,"score_lmetric":171326}],"chosen_score_linear":-85504.0,"chosen_score_lmetric":0,"t_first_token":282112.136057813,"t_first_token_unix":1779712977.1500018,"t_done":282112.313815637,"t_done_unix":1779712977.327759},{"request_id":"1279412:31:1391216:711","session_id":"1279412","input_length":101993,"t_proxy_recv":282110.259037973,"t_decision_unix":1779712975.2729795,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":101376,"affinity_cache_ratio":0.993950565234869,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":101376,"estimated_new_tokens":617,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":101993,"score_linear":101803.0,"score_lmetric":130580},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":13134,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1598,"cache_hit":0,"new_prefill":101993,"score_linear":13134.0,"score_lmetric":101993},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16550,"ongoing_decode_tokens":16550,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":0,"new_prefill":101993,"score_linear":16550.0,"score_lmetric":101993},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":169401,"ongoing_decode_tokens":106143,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":101993,"score_linear":169401.0,"score_lmetric":228102},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":37212,"ongoing_decode_tokens":37212,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":101993,"score_linear":37212.0,"score_lmetric":203986},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":12800,"new_prefill":89193,"score_linear":111464.0,"score_lmetric":369570},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":101993,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":61543,"ongoing_decode_tokens":61543,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1507,"cache_hit":101376,"new_prefill":617,"score_linear":-39833.0,"score_lmetric":617}],"chosen_score_linear":-39833.0,"chosen_score_lmetric":617,"t_first_token":282110.776734821,"t_first_token_unix":1779712975.7906785,"t_done":282114.721290723,"t_done_unix":1779712979.7352347},{"request_id":"1313181:6:1379623:639","session_id":"1313181","input_length":61543,"t_proxy_recv":282108.555093899,"t_decision_unix":1779712973.5690353,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":61440,"affinity_cache_ratio":0.9983263734299596,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":61440,"estimated_new_tokens":103,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":61543,"score_linear":101803.0,"score_lmetric":90130},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":13134,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1597,"cache_hit":0,"new_prefill":61543,"score_linear":13134.0,"score_lmetric":61543},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":0,"new_prefill":61543,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":61543,"score_linear":106143.0,"score_lmetric":61543},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":61543,"score_linear":33321.0,"score_lmetric":61543},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":93040,"num_requests":1,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":12800,"new_prefill":48743,"score_linear":80240.0,"score_lmetric":141783},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":61543,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101200,"ongoing_decode_tokens":101200,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":61440,"new_prefill":103,"score_linear":39760.0,"score_lmetric":103}],"chosen_score_linear":39760.0,"chosen_score_lmetric":103,"t_first_token":282108.671350814,"t_first_token_unix":1779712973.6852942,"t_done":282115.257734997,"t_done_unix":1779712980.2716784},{"request_id":"1362265:6:1399937:747","session_id":"1362265","input_length":10816,"t_proxy_recv":282115.812481839,"t_decision_unix":1779712980.8264234,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9940828402366864,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":10752,"estimated_new_tokens":64,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":6656,"new_prefill":4160,"score_linear":95147.0,"score_lmetric":32747},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":10752,"new_prefill":64,"score_linear":-10752.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2081,"cache_hit":0,"new_prefill":10816,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":169401,"ongoing_decode_tokens":106143,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":6656,"new_prefill":4160,"score_linear":162745.0,"score_lmetric":32436},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1769,"cache_hit":0,"new_prefill":10816,"score_linear":33321.0,"score_lmetric":10816},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":10816,"score_linear":124264.0,"score_lmetric":212816},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":10816,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":10816,"score_linear":121811.0,"score_lmetric":70675}],"chosen_score_linear":-10752.0,"chosen_score_lmetric":0,"t_first_token":282115.856683597,"t_first_token_unix":1779712980.8706264,"t_done":282116.092326671,"t_done_unix":1779712981.1062696},{"request_id":"1391404:2:1393919:720","session_id":"1391404","input_length":33321,"t_proxy_recv":282094.731789626,"t_decision_unix":1779712959.7457306,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":28160,"affinity_cache_ratio":0.8451126916959275,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":28160,"estimated_new_tokens":5161,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":2560,"new_prefill":30761,"score_linear":99243.0,"score_lmetric":59348},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1595,"cache_hit":0,"new_prefill":33321,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2012,"cache_hit":0,"new_prefill":33321,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":33321,"score_linear":15283.0,"score_lmetric":33321},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":28160,"new_prefill":5161,"score_linear":-28160.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":0,"new_prefill":33321,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":48944,"ongoing_decode_tokens":48944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1575,"cache_hit":0,"new_prefill":33321,"score_linear":48944.0,"score_lmetric":33321},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":33321,"score_linear":245313.0,"score_lmetric":235620}],"chosen_score_linear":-28160.0,"chosen_score_lmetric":0,"t_first_token":282095.851173802,"t_first_token_unix":1779712960.8651175,"t_done":282116.398883807,"t_done_unix":1779712981.4128273},{"request_id":"1399948:1:1399948:748","session_id":"1399948","input_length":4991,"t_proxy_recv":282115.848957285,"t_decision_unix":1779712980.8628986,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4991,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":4991,"score_linear":101803.0,"score_lmetric":33578},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":10816,"ongoing_decode_tokens":0,"pending_prefill_tokens":64,"num_requests":1,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":0,"new_prefill":4991,"score_linear":10816.0,"score_lmetric":5055},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2081,"cache_hit":0,"new_prefill":4991,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":169401,"ongoing_decode_tokens":106143,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":4991,"score_linear":169401.0,"score_lmetric":34098},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1769,"cache_hit":0,"new_prefill":4991,"score_linear":33321.0,"score_lmetric":4991},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":4991,"score_linear":124264.0,"score_lmetric":201166},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":4991,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":4991,"score_linear":121811.0,"score_lmetric":64850}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282116.170245426,"t_first_token_unix":1779712981.1841886,"t_done":282116.833671696,"t_done_unix":1779712981.847615},{"request_id":"1397840:2:1400216:749","session_id":"1397840","input_length":26440,"t_proxy_recv":282116.644093047,"t_decision_unix":1779712981.658034,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":16384,"affinity_cache_ratio":0.6196671709531013,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":16384,"estimated_new_tokens":10056,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":26440,"score_linear":101803.0,"score_lmetric":55027},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":0,"new_prefill":26440,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2081,"cache_hit":16384,"new_prefill":10056,"score_linear":-16384.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":169401,"ongoing_decode_tokens":106143,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":26440,"score_linear":169401.0,"score_lmetric":76996},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":0,"new_prefill":26440,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":26440,"score_linear":124264.0,"score_lmetric":244064},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":4991,"ongoing_decode_tokens":4991,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":26440,"score_linear":4991.0,"score_lmetric":26440},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":26440,"score_linear":121811.0,"score_lmetric":86299}],"chosen_score_linear":-16384.0,"chosen_score_lmetric":0,"t_first_token":282118.410437265,"t_first_token_unix":1779712983.424381,"t_done":282118.589934358,"t_done_unix":1779712983.6038775},{"request_id":"1398213:2:1400590:753","session_id":"1398213","input_length":11414,"t_proxy_recv":282117.947886074,"t_decision_unix":1779712982.9618273,"policy":"unified","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":3584,"affinity_cache_ratio":0.3140003504468197,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":3584,"estimated_new_tokens":7830,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":11414,"score_linear":101803.0,"score_lmetric":40001},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":101364,"ongoing_decode_tokens":3707,"pending_prefill_tokens":12153,"num_requests":2,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":0,"new_prefill":11414,"score_linear":101364.0,"score_lmetric":47134},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26440,"ongoing_decode_tokens":0,"pending_prefill_tokens":10056,"num_requests":1,"active_p_offloads":0,"cached_blocks":2081,"cache_hit":0,"new_prefill":11414,"score_linear":26440.0,"score_lmetric":21470},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":169401,"ongoing_decode_tokens":106143,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":11414,"score_linear":169401.0,"score_lmetric":46944},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":3584,"new_prefill":7830,"score_linear":-3584.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":11414,"score_linear":124264.0,"score_lmetric":214012},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":11414,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":11414,"score_linear":121811.0,"score_lmetric":71273}],"chosen_score_linear":-3584.0,"chosen_score_lmetric":0,"t_first_token":282118.636052633,"t_first_token_unix":1779712983.649996,"t_done":282119.03728095,"t_done_unix":1779712984.0512245},{"request_id":"1398213:3:1401507:759","session_id":"1398213","input_length":12910,"t_proxy_recv":282121.373385143,"t_decision_unix":1779712986.3873262,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":11264,"affinity_cache_ratio":0.8725019364833463,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":1646,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":12910,"score_linear":101803.0,"score_lmetric":41497},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":169473,"ongoing_decode_tokens":3707,"pending_prefill_tokens":29574,"num_requests":3,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":0,"new_prefill":12910,"score_linear":169473.0,"score_lmetric":127452},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2100,"cache_hit":0,"new_prefill":12910,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":169401,"ongoing_decode_tokens":106143,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":12910,"score_linear":169401.0,"score_lmetric":49936},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1796,"cache_hit":11264,"new_prefill":1646,"score_linear":-11264.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":12910,"score_linear":124264.0,"score_lmetric":217004},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":87072,"ongoing_decode_tokens":0,"pending_prefill_tokens":33824,"num_requests":2,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":12910,"score_linear":87072.0,"score_lmetric":93468},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":12910,"score_linear":121811.0,"score_lmetric":72769}],"chosen_score_linear":-11264.0,"chosen_score_lmetric":0,"t_first_token":282121.583730605,"t_first_token_unix":1779712986.5976746,"t_done":282124.185481241,"t_done_unix":1779712989.1994243},{"request_id":"1253743:8:1398344:743","session_id":"1253743","input_length":63258,"t_proxy_recv":282110.182628308,"t_decision_unix":1779712975.1965697,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":51200,"affinity_cache_ratio":0.8093837933542003,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":51200,"estimated_new_tokens":12058,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":63258,"score_linear":101803.0,"score_lmetric":91845},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":13134,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1598,"cache_hit":4608,"new_prefill":58650,"score_linear":8526.0,"score_lmetric":58650},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16550,"ongoing_decode_tokens":16550,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":0,"new_prefill":63258,"score_linear":16550.0,"score_lmetric":63258},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":51200,"new_prefill":12058,"score_linear":54943.0,"score_lmetric":12058},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":37212,"ongoing_decode_tokens":37212,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":63258,"score_linear":37212.0,"score_lmetric":126516},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":63258,"score_linear":124264.0,"score_lmetric":317700},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":63258,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":162939,"ongoing_decode_tokens":162939,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":63258,"score_linear":162939.0,"score_lmetric":126516}],"chosen_score_linear":54943.0,"chosen_score_lmetric":12058,"t_first_token":282122.716502873,"t_first_token_unix":1779712987.7304463,"t_done":282124.64467225,"t_done_unix":1779712989.658616},{"request_id":"1397840:3:1402568:761","session_id":"1397840","input_length":28603,"t_proxy_recv":282124.9904882,"t_decision_unix":1779712990.0044298,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":26112,"affinity_cache_ratio":0.9129112330874384,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":26112,"estimated_new_tokens":2491,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":28603,"score_linear":101803.0,"score_lmetric":57190},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":169473,"ongoing_decode_tokens":101364,"pending_prefill_tokens":17421,"num_requests":3,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":0,"new_prefill":28603,"score_linear":169473.0,"score_lmetric":138072},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2100,"cache_hit":26112,"new_prefill":2491,"score_linear":-26112.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":0,"new_prefill":28603,"score_linear":106143.0,"score_lmetric":28603},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":28603,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":28603,"score_linear":124264.0,"score_lmetric":248390},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":87072,"ongoing_decode_tokens":24442,"pending_prefill_tokens":13478,"num_requests":2,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":28603,"score_linear":87072.0,"score_lmetric":84162},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":28603,"score_linear":121811.0,"score_lmetric":88462}],"chosen_score_linear":-26112.0,"chosen_score_lmetric":0,"t_first_token":282125.476881637,"t_first_token_unix":1779712990.4908257,"t_done":282125.627169169,"t_done_unix":1779712990.6411126},{"request_id":"1403188:1:1403188:764","session_id":"1403188","input_length":8848,"t_proxy_recv":282127.389284393,"t_decision_unix":1779712992.4032257,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8848,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":8848,"score_linear":101803.0,"score_lmetric":37435},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":169473,"ongoing_decode_tokens":101364,"pending_prefill_tokens":17421,"num_requests":3,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":0,"new_prefill":8848,"score_linear":169473.0,"score_lmetric":78807},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2104,"cache_hit":0,"new_prefill":8848,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":0,"new_prefill":8848,"score_linear":106143.0,"score_lmetric":8848},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":8848,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":8848,"score_linear":124264.0,"score_lmetric":208880},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":87072,"ongoing_decode_tokens":87072,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":8848,"score_linear":87072.0,"score_lmetric":17696},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":8848,"score_linear":121811.0,"score_lmetric":68707}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282128.054509089,"t_first_token_unix":1779712993.0684536,"t_done":282128.6309976,"t_done_unix":1779712993.6449523},{"request_id":"1403989:1:1403989:767","session_id":"1403989","input_length":1954,"t_proxy_recv":282129.883240785,"t_decision_unix":1779712994.8971815,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1954,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":1954,"score_linear":101803.0,"score_lmetric":1954},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":169473,"ongoing_decode_tokens":101364,"pending_prefill_tokens":17421,"num_requests":3,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":0,"new_prefill":1954,"score_linear":169473.0,"score_lmetric":58125},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2104,"cache_hit":0,"new_prefill":1954,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":0,"new_prefill":1954,"score_linear":106143.0,"score_lmetric":1954},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":1954,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168412,"ongoing_decode_tokens":0,"pending_prefill_tokens":98268,"num_requests":3,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":1954,"score_linear":168412.0,"score_lmetric":300666},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":87072,"ongoing_decode_tokens":87072,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":1954,"score_linear":87072.0,"score_lmetric":3908},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":1954,"score_linear":121811.0,"score_lmetric":61813}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282130.012070586,"t_first_token_unix":1779712995.0260146,"t_done":282130.365139389,"t_done_unix":1779712995.3790824},{"request_id":"1400379:1:1400379:751","session_id":"1400379","input_length":3707,"t_proxy_recv":282117.160156008,"t_decision_unix":1779712982.174097,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3707,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":3707,"score_linear":101803.0,"score_lmetric":32294},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":0,"new_prefill":3707,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26440,"ongoing_decode_tokens":0,"pending_prefill_tokens":10056,"num_requests":1,"active_p_offloads":0,"cached_blocks":2081,"cache_hit":0,"new_prefill":3707,"score_linear":26440.0,"score_lmetric":13763},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":169401,"ongoing_decode_tokens":106143,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":3707,"score_linear":169401.0,"score_lmetric":31530},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":0,"new_prefill":3707,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":3707,"score_linear":124264.0,"score_lmetric":198598},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":3707,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":3707,"score_linear":121811.0,"score_lmetric":63566}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282117.387093403,"t_first_token_unix":1779712982.4010367,"t_done":282131.64895087,"t_done_unix":1779712996.6628942},{"request_id":"1253743:8:1404284:769","session_id":"1253743","input_length":63643,"t_proxy_recv":282130.909797125,"t_decision_unix":1779712995.923738,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":62976,"affinity_cache_ratio":0.9895196643778578,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":62976,"estimated_new_tokens":667,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":63643,"score_linear":101803.0,"score_lmetric":63643},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":169473,"ongoing_decode_tokens":101364,"pending_prefill_tokens":17421,"num_requests":3,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":4608,"new_prefill":59035,"score_linear":164865.0,"score_lmetric":229368},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2107,"cache_hit":0,"new_prefill":63643,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":62976,"new_prefill":667,"score_linear":43167.0,"score_lmetric":667},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":63643,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168412,"ongoing_decode_tokens":0,"pending_prefill_tokens":98268,"num_requests":3,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":63643,"score_linear":168412.0,"score_lmetric":485733},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":87072,"ongoing_decode_tokens":87072,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":63643,"score_linear":87072.0,"score_lmetric":127286},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":63643,"score_linear":121811.0,"score_lmetric":123502}],"chosen_score_linear":43167.0,"chosen_score_lmetric":667,"t_first_token":282131.133599196,"t_first_token_unix":1779712996.1475422,"t_done":282132.302101217,"t_done_unix":1779712997.3160448},{"request_id":"1366128:20:1400446:752","session_id":"1366128","input_length":97657,"t_proxy_recv":282117.413565293,"t_decision_unix":1779712982.4275067,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":85504,"affinity_cache_ratio":0.8755542357434695,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":85504,"estimated_new_tokens":12153,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":97657,"score_linear":101803.0,"score_lmetric":126244},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":3707,"ongoing_decode_tokens":3707,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":85504,"new_prefill":12153,"score_linear":-81797.0,"score_lmetric":12153},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26440,"ongoing_decode_tokens":0,"pending_prefill_tokens":10056,"num_requests":1,"active_p_offloads":0,"cached_blocks":2081,"cache_hit":0,"new_prefill":97657,"score_linear":26440.0,"score_lmetric":107713},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":169401,"ongoing_decode_tokens":106143,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":97657,"score_linear":169401.0,"score_lmetric":219430},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":0,"new_prefill":97657,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":97657,"score_linear":124264.0,"score_lmetric":386498},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":97657,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":97657,"score_linear":121811.0,"score_lmetric":157516}],"chosen_score_linear":-81797.0,"chosen_score_lmetric":12153,"t_first_token":282124.361295011,"t_first_token_unix":1779712989.3752387,"t_done":282132.379807186,"t_done_unix":1779712997.3937507},{"request_id":"1364090:5:1401054:756","session_id":"1364090","input_length":62630,"t_proxy_recv":282119.791542391,"t_decision_unix":1779712984.8054838,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":49152,"affinity_cache_ratio":0.7847996167970621,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":49152,"estimated_new_tokens":13478,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":7680,"new_prefill":54950,"score_linear":94123.0,"score_lmetric":83537},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":169473,"ongoing_decode_tokens":3707,"pending_prefill_tokens":29574,"num_requests":3,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":4096,"new_prefill":58534,"score_linear":165377.0,"score_lmetric":264324},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2100,"cache_hit":0,"new_prefill":62630,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":169401,"ongoing_decode_tokens":106143,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":7680,"new_prefill":54950,"score_linear":161721.0,"score_lmetric":134016},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1796,"cache_hit":0,"new_prefill":62630,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":62630,"score_linear":124264.0,"score_lmetric":316444},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24442,"ongoing_decode_tokens":0,"pending_prefill_tokens":20346,"num_requests":1,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":49152,"new_prefill":13478,"score_linear":-24710.0,"score_lmetric":33824},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":7680,"new_prefill":54950,"score_linear":114131.0,"score_lmetric":114809}],"chosen_score_linear":-24710.0,"chosen_score_lmetric":33824,"t_first_token":282127.011827557,"t_first_token_unix":1779712992.0257716,"t_done":282132.814828195,"t_done_unix":1779712997.8287716},{"request_id":"1400379:2:1401201:757","session_id":"1400379","input_length":3863,"t_proxy_recv":282131.652572209,"t_decision_unix":1779712996.6665137,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":3584,"affinity_cache_ratio":0.92777633963241,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":3584,"estimated_new_tokens":279,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":3863,"score_linear":101803.0,"score_lmetric":3863},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":165766,"ongoing_decode_tokens":165766,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":279,"score_linear":162182.0,"score_lmetric":558},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2107,"cache_hit":0,"new_prefill":3863,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":169786,"ongoing_decode_tokens":169786,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":0,"new_prefill":3863,"score_linear":169786.0,"score_lmetric":7726},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":0,"pending_prefill_tokens":21172,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":3863,"score_linear":21684.0,"score_lmetric":25035},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168412,"ongoing_decode_tokens":0,"pending_prefill_tokens":98268,"num_requests":3,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":3863,"score_linear":168412.0,"score_lmetric":306393},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":87072,"ongoing_decode_tokens":87072,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":3863,"score_linear":87072.0,"score_lmetric":7726},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":3863,"score_linear":121811.0,"score_lmetric":63722}],"chosen_score_linear":162182.0,"chosen_score_lmetric":558,"t_first_token":282131.717234405,"t_first_token_unix":1779712996.7311783,"t_done":282132.843585556,"t_done_unix":1779712997.8575287},{"request_id":"1366128:21:1402408:760","session_id":"1366128","input_length":97814,"t_proxy_recv":282132.431290808,"t_decision_unix":1779712997.4452322,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":97280,"affinity_cache_ratio":0.9945406588013985,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":97280,"estimated_new_tokens":534,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":97814,"score_linear":101803.0,"score_lmetric":97814},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":71972,"ongoing_decode_tokens":71972,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1654,"cache_hit":97280,"new_prefill":534,"score_linear":-25308.0,"score_lmetric":1068},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2107,"cache_hit":0,"new_prefill":97814,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":97814,"score_linear":106143.0,"score_lmetric":97814},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":0,"pending_prefill_tokens":21172,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":97814,"score_linear":21684.0,"score_lmetric":118986},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168412,"ongoing_decode_tokens":0,"pending_prefill_tokens":98268,"num_requests":3,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":97814,"score_linear":168412.0,"score_lmetric":588246},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":87072,"ongoing_decode_tokens":87072,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":97814,"score_linear":87072.0,"score_lmetric":195628},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":97814,"score_linear":121811.0,"score_lmetric":157673}],"chosen_score_linear":-25308.0,"chosen_score_lmetric":1068,"t_first_token":282132.650618194,"t_first_token_unix":1779712997.664562,"t_done":282133.267966395,"t_done_unix":1779712998.28191},{"request_id":"1363943:10:1401002:755","session_id":"1363943","input_length":24442,"t_proxy_recv":282119.558894707,"t_decision_unix":1779712984.5728357,"policy":"unified","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":6,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9845348171180754,"affinity_num_requests":3,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":4096,"estimated_new_tokens":20346,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":8704,"new_prefill":15738,"score_linear":93099.0,"score_lmetric":44325},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":169473,"ongoing_decode_tokens":3707,"pending_prefill_tokens":29574,"num_requests":3,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":24064,"new_prefill":378,"score_linear":145409.0,"score_lmetric":89856},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2100,"cache_hit":0,"new_prefill":24442,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":169401,"ongoing_decode_tokens":106143,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":4096,"new_prefill":20346,"score_linear":165305.0,"score_lmetric":64808},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1796,"cache_hit":0,"new_prefill":24442,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":24442,"score_linear":124264.0,"score_lmetric":240068},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":4096,"new_prefill":20346,"score_linear":-4096.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":4096,"new_prefill":20346,"score_linear":117715.0,"score_lmetric":80205}],"chosen_score_linear":-4096.0,"chosen_score_lmetric":0,"t_first_token":282123.571785325,"t_first_token_unix":1779712988.585729,"t_done":282133.346439815,"t_done_unix":1779712998.3603828},{"request_id":"1366128:22:1403837:766","session_id":"1366128","input_length":97917,"t_proxy_recv":282133.289828956,"t_decision_unix":1779712998.3037703,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":97792,"affinity_cache_ratio":0.9987234086011622,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":97792,"estimated_new_tokens":125,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":113019,"ongoing_decode_tokens":101803,"pending_prefill_tokens":11216,"num_requests":2,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":97917,"score_linear":113019.0,"score_lmetric":218266},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68109,"ongoing_decode_tokens":68109,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1655,"cache_hit":97792,"new_prefill":125,"score_linear":-29683.0,"score_lmetric":125},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":145650,"ongoing_decode_tokens":0,"pending_prefill_tokens":18674,"num_requests":3,"active_p_offloads":0,"cached_blocks":2107,"cache_hit":0,"new_prefill":97917,"score_linear":145650.0,"score_lmetric":349773},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":97917,"score_linear":106143.0,"score_lmetric":97917},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":0,"pending_prefill_tokens":21172,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":97917,"score_linear":21684.0,"score_lmetric":119089},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168412,"ongoing_decode_tokens":0,"pending_prefill_tokens":98268,"num_requests":3,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":97917,"score_linear":168412.0,"score_lmetric":588555},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24442,"ongoing_decode_tokens":24442,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1652,"cache_hit":0,"new_prefill":97917,"score_linear":24442.0,"score_lmetric":97917},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":97917,"score_linear":121811.0,"score_lmetric":157776}],"chosen_score_linear":-29683.0,"chosen_score_lmetric":125,"t_first_token":282133.466930945,"t_first_token_unix":1779712998.4808743,"t_done":282133.582958894,"t_done_unix":1779712998.596902},{"request_id":"1400379:3:1402972:763","session_id":"1400379","input_length":11216,"t_proxy_recv":282132.850510322,"t_decision_unix":1779712997.864452,"policy":"unified","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":0,"affinity_cache_hit":3584,"affinity_cache_ratio":0.3195435092724679,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":11216,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":11216,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":11216,"score_linear":101803.0,"score_lmetric":11216},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":165923,"ongoing_decode_tokens":165923,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1654,"cache_hit":3584,"new_prefill":7632,"score_linear":162339.0,"score_lmetric":15264},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":21009,"ongoing_decode_tokens":0,"pending_prefill_tokens":13329,"num_requests":1,"active_p_offloads":0,"cached_blocks":2107,"cache_hit":0,"new_prefill":11216,"score_linear":21009.0,"score_lmetric":24545},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":11216,"score_linear":106143.0,"score_lmetric":11216},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":0,"pending_prefill_tokens":21172,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":11216,"score_linear":21684.0,"score_lmetric":32388},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168412,"ongoing_decode_tokens":0,"pending_prefill_tokens":98268,"num_requests":3,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":11216,"score_linear":168412.0,"score_lmetric":328452},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24442,"ongoing_decode_tokens":24442,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1652,"cache_hit":0,"new_prefill":11216,"score_linear":24442.0,"score_lmetric":11216},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":11216,"score_linear":121811.0,"score_lmetric":71075}],"chosen_score_linear":101803.0,"chosen_score_lmetric":11216,"t_first_token":282133.799032305,"t_first_token_unix":1779712998.812976,"t_done":282134.34747281,"t_done_unix":1779712999.361416},{"request_id":"1381387:4:1400724:754","session_id":"1381387","input_length":68109,"t_proxy_recv":282118.525378174,"t_decision_unix":1779712983.5393195,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":50688,"affinity_cache_ratio":0.7442188257058538,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":50688,"estimated_new_tokens":17421,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":68109,"score_linear":101803.0,"score_lmetric":96696},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":101364,"ongoing_decode_tokens":3707,"pending_prefill_tokens":12153,"num_requests":2,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":50688,"new_prefill":17421,"score_linear":50676.0,"score_lmetric":59148},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26440,"ongoing_decode_tokens":26440,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2081,"cache_hit":0,"new_prefill":68109,"score_linear":26440.0,"score_lmetric":68109},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":169401,"ongoing_decode_tokens":106143,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":68109,"score_linear":169401.0,"score_lmetric":160334},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":11414,"ongoing_decode_tokens":0,"pending_prefill_tokens":7830,"num_requests":1,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":0,"new_prefill":68109,"score_linear":11414.0,"score_lmetric":75939},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":68109,"score_linear":124264.0,"score_lmetric":327402},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":68109,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":68109,"score_linear":121811.0,"score_lmetric":127968}],"chosen_score_linear":50676.0,"chosen_score_lmetric":59148,"t_first_token":282130.946963926,"t_first_token_unix":1779712995.960907,"t_done":282134.673673785,"t_done_unix":1779712999.687618},{"request_id":"1405199:1:1405199:776","session_id":"1405199","input_length":11503,"t_proxy_recv":282134.136010765,"t_decision_unix":1779712999.149941,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":11503,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":11503,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":113019,"ongoing_decode_tokens":113019,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":11503,"score_linear":113019.0,"score_lmetric":23006},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68109,"ongoing_decode_tokens":68109,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1655,"cache_hit":0,"new_prefill":11503,"score_linear":68109.0,"score_lmetric":11503},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":145650,"ongoing_decode_tokens":21009,"pending_prefill_tokens":5345,"num_requests":3,"active_p_offloads":0,"cached_blocks":2107,"cache_hit":0,"new_prefill":11503,"score_linear":145650.0,"score_lmetric":50544},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":11503,"score_linear":106143.0,"score_lmetric":11503},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":11503,"score_linear":21684.0,"score_lmetric":11503},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168412,"ongoing_decode_tokens":0,"pending_prefill_tokens":98268,"num_requests":3,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":11503,"score_linear":168412.0,"score_lmetric":329313},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32944,"ongoing_decode_tokens":0,"pending_prefill_tokens":32944,"num_requests":1,"active_p_offloads":0,"cached_blocks":1691,"cache_hit":0,"new_prefill":11503,"score_linear":32944.0,"score_lmetric":44447},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":11503,"score_linear":121811.0,"score_lmetric":71362}],"chosen_score_linear":106143.0,"chosen_score_lmetric":11503,"t_first_token":282135.091397598,"t_first_token_unix":1779713000.105341,"t_done":282135.668902177,"t_done_unix":1779713000.6828456},{"request_id":"1398594:2:1404702:771","session_id":"1398594","input_length":21009,"t_proxy_recv":282132.435046669,"t_decision_unix":1779712997.448988,"policy":"unified","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":7680,"affinity_cache_ratio":0.3655576181636441,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":13329,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":21009,"score_linear":101803.0,"score_lmetric":21009},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":169786,"ongoing_decode_tokens":71972,"pending_prefill_tokens":534,"num_requests":3,"active_p_offloads":0,"cached_blocks":1654,"cache_hit":0,"new_prefill":21009,"score_linear":169786.0,"score_lmetric":64629},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2107,"cache_hit":7680,"new_prefill":13329,"score_linear":-7680.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":21009,"score_linear":106143.0,"score_lmetric":21009},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":0,"pending_prefill_tokens":21172,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":21009,"score_linear":21684.0,"score_lmetric":42181},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168412,"ongoing_decode_tokens":0,"pending_prefill_tokens":98268,"num_requests":3,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":21009,"score_linear":168412.0,"score_lmetric":357831},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":87072,"ongoing_decode_tokens":87072,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":21009,"score_linear":87072.0,"score_lmetric":42018},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":21009,"score_linear":121811.0,"score_lmetric":80868}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":282134.126874087,"t_first_token_unix":1779712999.140818,"t_done":282138.721485879,"t_done_unix":1779713003.7354293},{"request_id":"1399948:2:1405276:778","session_id":"1399948","input_length":6149,"t_proxy_recv":282134.354575942,"t_decision_unix":1779712999.3685174,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":4608,"affinity_cache_ratio":0.7493901447389819,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":1541,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":215061,"ongoing_decode_tokens":101803,"pending_prefill_tokens":3178,"num_requests":2,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":6149,"score_linear":215061.0,"score_lmetric":18654},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68109,"ongoing_decode_tokens":68109,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1655,"cache_hit":0,"new_prefill":6149,"score_linear":68109.0,"score_lmetric":6149},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":145650,"ongoing_decode_tokens":21009,"pending_prefill_tokens":5345,"num_requests":3,"active_p_offloads":0,"cached_blocks":2107,"cache_hit":0,"new_prefill":6149,"score_linear":145650.0,"score_lmetric":34482},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":117646,"ongoing_decode_tokens":106143,"pending_prefill_tokens":11503,"num_requests":2,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":6149,"score_linear":117646.0,"score_lmetric":35304},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":6149,"score_linear":21684.0,"score_lmetric":6149},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168412,"ongoing_decode_tokens":0,"pending_prefill_tokens":98268,"num_requests":3,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":6149,"score_linear":168412.0,"score_lmetric":313251},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32944,"ongoing_decode_tokens":0,"pending_prefill_tokens":32944,"num_requests":1,"active_p_offloads":0,"cached_blocks":1691,"cache_hit":4608,"new_prefill":1541,"score_linear":28336.0,"score_lmetric":34485},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":6149,"score_linear":121811.0,"score_lmetric":66008}],"chosen_score_linear":28336.0,"chosen_score_lmetric":34485,"t_first_token":282138.694860833,"t_first_token_unix":1779713003.7088044,"t_done":282138.744635803,"t_done_unix":1779713003.758579},{"request_id":"1382089:2:1404847:773","session_id":"1382089","input_length":39203,"t_proxy_recv":282132.901981435,"t_decision_unix":1779712997.9159229,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":37888,"affinity_cache_ratio":0.9664566487258628,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":37888,"estimated_new_tokens":1315,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":113019,"ongoing_decode_tokens":101803,"pending_prefill_tokens":11216,"num_requests":2,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":39203,"score_linear":113019.0,"score_lmetric":100838},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":165923,"ongoing_decode_tokens":165923,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1654,"cache_hit":0,"new_prefill":39203,"score_linear":165923.0,"score_lmetric":78406},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":21009,"ongoing_decode_tokens":0,"pending_prefill_tokens":13329,"num_requests":1,"active_p_offloads":0,"cached_blocks":2107,"cache_hit":37888,"new_prefill":1315,"score_linear":-16879.0,"score_lmetric":14644},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":39203,"score_linear":106143.0,"score_lmetric":39203},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":0,"pending_prefill_tokens":21172,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":39203,"score_linear":21684.0,"score_lmetric":60375},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168412,"ongoing_decode_tokens":0,"pending_prefill_tokens":98268,"num_requests":3,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":39203,"score_linear":168412.0,"score_lmetric":412413},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24442,"ongoing_decode_tokens":24442,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1652,"cache_hit":0,"new_prefill":39203,"score_linear":24442.0,"score_lmetric":39203},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":34816,"new_prefill":4387,"score_linear":86995.0,"score_lmetric":64246}],"chosen_score_linear":-16879.0,"chosen_score_lmetric":14644,"t_first_token":282138.026371521,"t_first_token_unix":1779713003.0403154,"t_done":282138.893588677,"t_done_unix":1779713003.9075325},{"request_id":"1399948:3:1405988:782","session_id":"1399948","input_length":7995,"t_proxy_recv":282138.748356703,"t_decision_unix":1779713003.7622983,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":6144,"affinity_cache_ratio":0.7684803001876173,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":6144,"estimated_new_tokens":1851,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":235027,"ongoing_decode_tokens":101803,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":7995,"score_linear":235027.0,"score_lmetric":61161},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":179082,"ongoing_decode_tokens":69508,"pending_prefill_tokens":11782,"num_requests":2,"active_p_offloads":0,"cached_blocks":1689,"cache_hit":0,"new_prefill":7995,"score_linear":179082.0,"score_lmetric":39554},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":124641,"ongoing_decode_tokens":124641,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2133,"cache_hit":0,"new_prefill":7995,"score_linear":124641.0,"score_lmetric":15990},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":7995,"score_linear":106143.0,"score_lmetric":7995},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":7995,"score_linear":21684.0,"score_lmetric":7995},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":288389,"ongoing_decode_tokens":124264,"pending_prefill_tokens":6429,"num_requests":4,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":7995,"score_linear":288389.0,"score_lmetric":57696},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32944,"ongoing_decode_tokens":32944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1694,"cache_hit":6144,"new_prefill":1851,"score_linear":26800.0,"score_lmetric":1851},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":7995,"score_linear":121811.0,"score_lmetric":67854}],"chosen_score_linear":26800.0,"chosen_score_lmetric":1851,"t_first_token":282138.956124142,"t_first_token_unix":1779713003.9700677,"t_done":282139.387988635,"t_done_unix":1779713004.4019318},{"request_id":"1373577:3:1404917:774","session_id":"1373577","input_length":85438,"t_proxy_recv":282133.198895369,"t_decision_unix":1779712998.2128365,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":81408,"affinity_cache_ratio":0.9528312928673424,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":81408,"estimated_new_tokens":4030,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":113019,"ongoing_decode_tokens":101803,"pending_prefill_tokens":11216,"num_requests":2,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":85438,"score_linear":113019.0,"score_lmetric":193308},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":165923,"ongoing_decode_tokens":165923,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1654,"cache_hit":0,"new_prefill":85438,"score_linear":165923.0,"score_lmetric":170876},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":60212,"ongoing_decode_tokens":0,"pending_prefill_tokens":14644,"num_requests":2,"active_p_offloads":0,"cached_blocks":2107,"cache_hit":81408,"new_prefill":4030,"score_linear":-21196.0,"score_lmetric":37348},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":85438,"score_linear":106143.0,"score_lmetric":85438},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":0,"pending_prefill_tokens":21172,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":85438,"score_linear":21684.0,"score_lmetric":106610},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168412,"ongoing_decode_tokens":0,"pending_prefill_tokens":98268,"num_requests":3,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":85438,"score_linear":168412.0,"score_lmetric":551118},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24442,"ongoing_decode_tokens":24442,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1652,"cache_hit":2560,"new_prefill":82878,"score_linear":21882.0,"score_lmetric":82878},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":85438,"score_linear":121811.0,"score_lmetric":145297}],"chosen_score_linear":-21196.0,"chosen_score_lmetric":37348,"t_first_token":282138.354788943,"t_first_token_unix":1779713003.3687327,"t_done":282140.887643136,"t_done_unix":1779713005.9015868},{"request_id":"1406966:1:1406966:785","session_id":"1406966","input_length":2115,"t_proxy_recv":282140.382790099,"t_decision_unix":1779713005.3967314,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":2115,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":2115,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":235027,"ongoing_decode_tokens":101803,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":2115,"score_linear":235027.0,"score_lmetric":43521},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":179082,"ongoing_decode_tokens":69508,"pending_prefill_tokens":11782,"num_requests":2,"active_p_offloads":0,"cached_blocks":1689,"cache_hit":0,"new_prefill":2115,"score_linear":179082.0,"score_lmetric":27794},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":85438,"ongoing_decode_tokens":85438,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":0,"new_prefill":2115,"score_linear":85438.0,"score_lmetric":2115},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":2115,"score_linear":106143.0,"score_lmetric":2115},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":2115,"score_linear":21684.0,"score_lmetric":2115},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":288389,"ongoing_decode_tokens":124264,"pending_prefill_tokens":6429,"num_requests":4,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":2115,"score_linear":288389.0,"score_lmetric":34176},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32944,"ongoing_decode_tokens":32944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1697,"cache_hit":0,"new_prefill":2115,"score_linear":32944.0,"score_lmetric":2115},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":2115,"score_linear":121811.0,"score_lmetric":61974}],"chosen_score_linear":32944.0,"chosen_score_lmetric":2115,"t_first_token":282140.564466561,"t_first_token_unix":1779713005.5784101,"t_done":282140.929261976,"t_done_unix":1779713005.9432049},{"request_id":"1404395:1:1404395:770","session_id":"1404395","input_length":21684,"t_proxy_recv":282131.301101853,"t_decision_unix":1779712996.3150437,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":512,"estimated_new_tokens":21172,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":512,"new_prefill":21172,"score_linear":101291.0,"score_lmetric":21172},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":169473,"ongoing_decode_tokens":169473,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":0,"new_prefill":21684,"score_linear":169473.0,"score_lmetric":65052},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2107,"cache_hit":0,"new_prefill":21684,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":169786,"ongoing_decode_tokens":169786,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":0,"new_prefill":21684,"score_linear":169786.0,"score_lmetric":43368},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":512,"new_prefill":21172,"score_linear":-512.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168412,"ongoing_decode_tokens":0,"pending_prefill_tokens":98268,"num_requests":3,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":21684,"score_linear":168412.0,"score_lmetric":359856},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":87072,"ongoing_decode_tokens":87072,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":21684,"score_linear":87072.0,"score_lmetric":43368},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":21684,"score_linear":121811.0,"score_lmetric":81543}],"chosen_score_linear":-512.0,"chosen_score_lmetric":0,"t_first_token":282133.619566876,"t_first_token_unix":1779712998.6335099,"t_done":282141.148875467,"t_done_unix":1779713006.1628187},{"request_id":"1399948:4:1407140:786","session_id":"1399948","input_length":8540,"t_proxy_recv":282140.915487229,"t_decision_unix":1779713005.9294288,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":7680,"affinity_cache_ratio":0.8992974238875878,"affinity_num_requests":2,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":860,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":235027,"ongoing_decode_tokens":101803,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":8540,"score_linear":235027.0,"score_lmetric":62796},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":179082,"ongoing_decode_tokens":69508,"pending_prefill_tokens":11782,"num_requests":2,"active_p_offloads":0,"cached_blocks":1689,"cache_hit":0,"new_prefill":8540,"score_linear":179082.0,"score_lmetric":40644},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2143,"cache_hit":0,"new_prefill":8540,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":8540,"score_linear":106143.0,"score_lmetric":8540},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":8540,"score_linear":21684.0,"score_lmetric":8540},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":288389,"ongoing_decode_tokens":124264,"pending_prefill_tokens":6429,"num_requests":4,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":8540,"score_linear":288389.0,"score_lmetric":59876},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":35059,"ongoing_decode_tokens":35059,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1697,"cache_hit":7680,"new_prefill":860,"score_linear":27379.0,"score_lmetric":1720},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":8540,"score_linear":121811.0,"score_lmetric":68399}],"chosen_score_linear":27379.0,"chosen_score_lmetric":1720,"t_first_token":282141.011757657,"t_first_token_unix":1779713006.025701,"t_done":282141.181537149,"t_done_unix":1779713006.19548},{"request_id":"1407494:1:1407494:787","session_id":"1407494","input_length":1531,"t_proxy_recv":282142.179663556,"t_decision_unix":1779713007.1936047,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1531,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":235027,"ongoing_decode_tokens":101803,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":1531,"score_linear":235027.0,"score_lmetric":41769},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":179082,"ongoing_decode_tokens":69508,"pending_prefill_tokens":11782,"num_requests":2,"active_p_offloads":0,"cached_blocks":1689,"cache_hit":0,"new_prefill":1531,"score_linear":179082.0,"score_lmetric":26626},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2143,"cache_hit":0,"new_prefill":1531,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":1531,"score_linear":106143.0,"score_lmetric":1531},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1860,"cache_hit":0,"new_prefill":1531,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":288389,"ongoing_decode_tokens":124264,"pending_prefill_tokens":6429,"num_requests":4,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":1531,"score_linear":288389.0,"score_lmetric":31840},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32944,"ongoing_decode_tokens":32944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1702,"cache_hit":0,"new_prefill":1531,"score_linear":32944.0,"score_lmetric":1531},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":1531,"score_linear":121811.0,"score_lmetric":61390}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282142.286030847,"t_first_token_unix":1779713007.2999744,"t_done":282142.556086477,"t_done_unix":1779713007.5700295},{"request_id":"1406966:2:1407710:789","session_id":"1406966","input_length":4789,"t_proxy_recv":282142.862536065,"t_decision_unix":1779713007.876477,"policy":"unified","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":2,"affinity_cache_hit":2048,"affinity_cache_ratio":0.42764669033201086,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":4789,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":235027,"ongoing_decode_tokens":101803,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":4789,"score_linear":235027.0,"score_lmetric":51543},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":179082,"ongoing_decode_tokens":69508,"pending_prefill_tokens":11782,"num_requests":2,"active_p_offloads":0,"cached_blocks":1689,"cache_hit":0,"new_prefill":4789,"score_linear":179082.0,"score_lmetric":33142},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":4789,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":4789,"score_linear":106143.0,"score_lmetric":4789},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":1501,"ongoing_decode_tokens":1501,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1860,"cache_hit":0,"new_prefill":4789,"score_linear":1501.0,"score_lmetric":4789},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":288389,"ongoing_decode_tokens":124264,"pending_prefill_tokens":6429,"num_requests":4,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":4789,"score_linear":288389.0,"score_lmetric":44872},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32944,"ongoing_decode_tokens":32944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1702,"cache_hit":2048,"new_prefill":2741,"score_linear":30896.0,"score_lmetric":2741},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":4789,"score_linear":121811.0,"score_lmetric":64648}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282143.163377175,"t_first_token_unix":1779713008.1773207,"t_done":282143.386153092,"t_done_unix":1779713008.400096},{"request_id":"1407599:1:1407599:788","session_id":"1407599","input_length":1501,"t_proxy_recv":282142.534633506,"t_decision_unix":1779713007.5485747,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":1501,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":235027,"ongoing_decode_tokens":101803,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":1501,"score_linear":235027.0,"score_lmetric":41679},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":179082,"ongoing_decode_tokens":69508,"pending_prefill_tokens":11782,"num_requests":2,"active_p_offloads":0,"cached_blocks":1689,"cache_hit":0,"new_prefill":1501,"score_linear":179082.0,"score_lmetric":26566},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":1531,"ongoing_decode_tokens":1531,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2143,"cache_hit":0,"new_prefill":1501,"score_linear":1531.0,"score_lmetric":1501},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":1501,"score_linear":106143.0,"score_lmetric":1501},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1860,"cache_hit":0,"new_prefill":1501,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":288389,"ongoing_decode_tokens":124264,"pending_prefill_tokens":6429,"num_requests":4,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":1501,"score_linear":288389.0,"score_lmetric":31720},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32944,"ongoing_decode_tokens":32944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1702,"cache_hit":0,"new_prefill":1501,"score_linear":32944.0,"score_lmetric":1501},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":1501,"score_linear":121811.0,"score_lmetric":61360}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282142.638085554,"t_first_token_unix":1779713007.6520298,"t_done":282143.604274724,"t_done_unix":1779713008.6182184},{"request_id":"1399948:5:1407922:791","session_id":"1399948","input_length":8612,"t_proxy_recv":282143.701381353,"t_decision_unix":1779713008.715323,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":8192,"affinity_cache_ratio":0.9512308406874129,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":8192,"estimated_new_tokens":420,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":235027,"ongoing_decode_tokens":101803,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":8612,"score_linear":235027.0,"score_lmetric":63012},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":179082,"ongoing_decode_tokens":69508,"pending_prefill_tokens":11782,"num_requests":2,"active_p_offloads":0,"cached_blocks":1689,"cache_hit":0,"new_prefill":8612,"score_linear":179082.0,"score_lmetric":40788},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2155,"cache_hit":0,"new_prefill":8612,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":8612,"score_linear":106143.0,"score_lmetric":8612},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1863,"cache_hit":0,"new_prefill":8612,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":124264,"pending_prefill_tokens":11009,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":8612,"score_linear":361577.0,"score_lmetric":98105},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32944,"ongoing_decode_tokens":32944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1702,"cache_hit":8192,"new_prefill":420,"score_linear":24752.0,"score_lmetric":420},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":8612,"score_linear":121811.0,"score_lmetric":68471}],"chosen_score_linear":24752.0,"chosen_score_lmetric":420,"t_first_token":282143.750510156,"t_first_token_unix":1779713008.7644534,"t_done":282143.92539775,"t_done_unix":1779713008.9393408},{"request_id":"1405155:1:1405155:775","session_id":"1405155","input_length":32944,"t_proxy_recv":282134.003963031,"t_decision_unix":1779712999.0179043,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":32944,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":113019,"ongoing_decode_tokens":113019,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":32944,"score_linear":113019.0,"score_lmetric":65888},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68109,"ongoing_decode_tokens":68109,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1655,"cache_hit":512,"new_prefill":32432,"score_linear":67597.0,"score_lmetric":32432},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":145650,"ongoing_decode_tokens":0,"pending_prefill_tokens":18674,"num_requests":3,"active_p_offloads":0,"cached_blocks":2107,"cache_hit":0,"new_prefill":32944,"score_linear":145650.0,"score_lmetric":154854},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":32944,"score_linear":106143.0,"score_lmetric":32944},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":32944,"score_linear":21684.0,"score_lmetric":32944},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168412,"ongoing_decode_tokens":0,"pending_prefill_tokens":98268,"num_requests":3,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":32944,"score_linear":168412.0,"score_lmetric":393636},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1691,"cache_hit":0,"new_prefill":32944,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":32944,"score_linear":121811.0,"score_lmetric":92803}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282138.691393907,"t_first_token_unix":1779713003.7053378,"t_done":282144.335143709,"t_done_unix":1779713009.349087},{"request_id":"1407995:1:1407995:793","session_id":"1407995","input_length":2944,"t_proxy_recv":282143.994273217,"t_decision_unix":1779713009.0082147,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":2944,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":2944,"score_linear":262703.0,"score_lmetric":83984},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":179082,"ongoing_decode_tokens":69508,"pending_prefill_tokens":11782,"num_requests":2,"active_p_offloads":0,"cached_blocks":1689,"cache_hit":0,"new_prefill":2944,"score_linear":179082.0,"score_lmetric":29452},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2155,"cache_hit":0,"new_prefill":2944,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":2944,"score_linear":106143.0,"score_lmetric":2944},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1863,"cache_hit":0,"new_prefill":2944,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":124264,"pending_prefill_tokens":11009,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":2944,"score_linear":361577.0,"score_lmetric":69765},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32944,"ongoing_decode_tokens":32944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1702,"cache_hit":0,"new_prefill":2944,"score_linear":32944.0,"score_lmetric":2944},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":2944,"score_linear":121811.0,"score_lmetric":62803}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282144.174797476,"t_first_token_unix":1779713009.1887405,"t_done":282144.559419289,"t_done_unix":1779713009.5733643},{"request_id":"1381387:5:1404170:768","session_id":"1381387","input_length":69508,"t_proxy_recv":282134.685889368,"t_decision_unix":1779712999.6998308,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":68096,"affinity_cache_ratio":0.9796857915635611,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":68096,"estimated_new_tokens":1412,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":215061,"ongoing_decode_tokens":101803,"pending_prefill_tokens":3178,"num_requests":2,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":69508,"score_linear":215061.0,"score_lmetric":145372},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1689,"cache_hit":68096,"new_prefill":1412,"score_linear":-68096.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":145650,"ongoing_decode_tokens":21009,"pending_prefill_tokens":5345,"num_requests":3,"active_p_offloads":0,"cached_blocks":2107,"cache_hit":0,"new_prefill":69508,"score_linear":145650.0,"score_lmetric":224559},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":117646,"ongoing_decode_tokens":106143,"pending_prefill_tokens":11503,"num_requests":2,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":69508,"score_linear":117646.0,"score_lmetric":162022},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":69508,"score_linear":21684.0,"score_lmetric":69508},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168412,"ongoing_decode_tokens":0,"pending_prefill_tokens":98268,"num_requests":3,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":69508,"score_linear":168412.0,"score_lmetric":503328},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":39093,"ongoing_decode_tokens":0,"pending_prefill_tokens":34485,"num_requests":2,"active_p_offloads":0,"cached_blocks":1691,"cache_hit":0,"new_prefill":69508,"score_linear":39093.0,"score_lmetric":207986},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":69508,"score_linear":121811.0,"score_lmetric":129367}],"chosen_score_linear":-68096.0,"chosen_score_lmetric":0,"t_first_token":282135.313876801,"t_first_token_unix":1779713000.3278213,"t_done":282144.604928537,"t_done_unix":1779713009.6188722},{"request_id":"1366128:23:1405900:781","session_id":"1366128","input_length":109574,"t_proxy_recv":282136.651617211,"t_decision_unix":1779713001.6655583,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":97792,"affinity_cache_ratio":0.8924744921240441,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":97792,"estimated_new_tokens":11782,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":235027,"ongoing_decode_tokens":101803,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":109574,"score_linear":235027.0,"score_lmetric":365898},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":69508,"ongoing_decode_tokens":69508,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1689,"cache_hit":97792,"new_prefill":11782,"score_linear":-28284.0,"score_lmetric":11782},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":145650,"ongoing_decode_tokens":21009,"pending_prefill_tokens":5345,"num_requests":3,"active_p_offloads":0,"cached_blocks":2107,"cache_hit":0,"new_prefill":109574,"score_linear":145650.0,"score_lmetric":344757},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":109574,"score_linear":106143.0,"score_lmetric":109574},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":109574,"score_linear":21684.0,"score_lmetric":109574},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":288389,"ongoing_decode_tokens":124264,"pending_prefill_tokens":6429,"num_requests":4,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":109574,"score_linear":288389.0,"score_lmetric":464012},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":39093,"ongoing_decode_tokens":0,"pending_prefill_tokens":34485,"num_requests":2,"active_p_offloads":0,"cached_blocks":1691,"cache_hit":0,"new_prefill":109574,"score_linear":39093.0,"score_lmetric":288118},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":109574,"score_linear":121811.0,"score_lmetric":169433}],"chosen_score_linear":-28284.0,"chosen_score_lmetric":11782,"t_first_token":282144.240192002,"t_first_token_unix":1779713009.2541358,"t_done":282145.491769392,"t_done_unix":1779713010.5057132},{"request_id":"1340290:7:1408176:795","session_id":"1340290","input_length":27786,"t_proxy_recv":282144.684194834,"t_decision_unix":1779713009.6981363,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9766069243503923,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":27136,"estimated_new_tokens":650,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":27786,"score_linear":262703.0,"score_lmetric":183352},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":109574,"ongoing_decode_tokens":109574,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1692,"cache_hit":0,"new_prefill":27786,"score_linear":109574.0,"score_lmetric":27786},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2155,"cache_hit":0,"new_prefill":27786,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":27786,"score_linear":106143.0,"score_lmetric":27786},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1868,"cache_hit":0,"new_prefill":27786,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":27786,"score_linear":361577.0,"score_lmetric":180595},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":27136,"new_prefill":650,"score_linear":-27136.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":27786,"score_linear":121811.0,"score_lmetric":87645}],"chosen_score_linear":-27136.0,"chosen_score_lmetric":0,"t_first_token":282144.848922894,"t_first_token_unix":1779713009.8628664,"t_done":282145.544586889,"t_done_unix":1779713010.5585299},{"request_id":"1403188:2:1408249:796","session_id":"1403188","input_length":14728,"t_proxy_recv":282144.943512942,"t_decision_unix":1779713009.9574544,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":8704,"affinity_cache_ratio":0.5909831613253667,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":8704,"estimated_new_tokens":6024,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":14728,"score_linear":262703.0,"score_lmetric":131120},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":109574,"ongoing_decode_tokens":109574,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1692,"cache_hit":0,"new_prefill":14728,"score_linear":109574.0,"score_lmetric":14728},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2155,"cache_hit":0,"new_prefill":14728,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":14728,"score_linear":106143.0,"score_lmetric":14728},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1868,"cache_hit":8704,"new_prefill":6024,"score_linear":-8704.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":14728,"score_linear":361577.0,"score_lmetric":115305},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":27786,"ongoing_decode_tokens":27786,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":0,"new_prefill":14728,"score_linear":27786.0,"score_lmetric":14728},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":14728,"score_linear":121811.0,"score_lmetric":74587}],"chosen_score_linear":-8704.0,"chosen_score_lmetric":0,"t_first_token":282145.62146371,"t_first_token_unix":1779713010.6354084,"t_done":282145.913812183,"t_done_unix":1779713010.9277554},{"request_id":"1366128:24:1407996:794","session_id":"1366128","input_length":109775,"t_proxy_recv":282145.51026135,"t_decision_unix":1779713010.5242023,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":109568,"affinity_cache_ratio":0.998114324755181,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":109568,"estimated_new_tokens":207,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":109775,"score_linear":262703.0,"score_lmetric":511308},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1715,"cache_hit":109568,"new_prefill":207,"score_linear":-109568.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2155,"cache_hit":0,"new_prefill":109775,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":109775,"score_linear":106143.0,"score_lmetric":109775},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":14728,"ongoing_decode_tokens":0,"pending_prefill_tokens":6024,"num_requests":1,"active_p_offloads":0,"cached_blocks":1868,"cache_hit":0,"new_prefill":109775,"score_linear":14728.0,"score_lmetric":115799},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":109775,"score_linear":361577.0,"score_lmetric":590540},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":27786,"ongoing_decode_tokens":27786,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":0,"new_prefill":109775,"score_linear":27786.0,"score_lmetric":109775},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":109775,"score_linear":121811.0,"score_lmetric":169634}],"chosen_score_linear":-109568.0,"chosen_score_lmetric":0,"t_first_token":282145.719135044,"t_first_token_unix":1779713010.733078,"t_done":282146.073252226,"t_done_unix":1779713011.0871956},{"request_id":"1408889:1:1408889:804","session_id":"1408889","input_length":3230,"t_proxy_recv":282146.996142077,"t_decision_unix":1779713012.0100834,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":3230,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3230,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":3230,"score_linear":262703.0,"score_lmetric":85128},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":11070,"ongoing_decode_tokens":11070,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1715,"cache_hit":0,"new_prefill":3230,"score_linear":11070.0,"score_lmetric":3230},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98371,"ongoing_decode_tokens":7553,"pending_prefill_tokens":5314,"num_requests":2,"active_p_offloads":0,"cached_blocks":2155,"cache_hit":0,"new_prefill":3230,"score_linear":98371.0,"score_lmetric":17088},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":3230,"score_linear":106143.0,"score_lmetric":3230},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":6377,"ongoing_decode_tokens":6377,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":0,"new_prefill":3230,"score_linear":6377.0,"score_lmetric":3230},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":3230,"score_linear":361577.0,"score_lmetric":57815},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":75193,"ongoing_decode_tokens":1801,"pending_prefill_tokens":10416,"num_requests":2,"active_p_offloads":0,"cached_blocks":1768,"cache_hit":0,"new_prefill":3230,"score_linear":75193.0,"score_lmetric":27292},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":3230,"score_linear":121811.0,"score_lmetric":63089}],"chosen_score_linear":11070.0,"chosen_score_lmetric":3230,"t_first_token":282147.196474561,"t_first_token_unix":1779713012.2104175,"t_done":282147.362051318,"t_done_unix":1779713012.3759944},{"request_id":"1362265:7:1408772:800","session_id":"1362265","input_length":11070,"t_proxy_recv":282146.609680808,"t_decision_unix":1779713011.6236215,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9712737127371274,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":10752,"estimated_new_tokens":318,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":6656,"new_prefill":4414,"score_linear":256047.0,"score_lmetric":89864},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1715,"cache_hit":10752,"new_prefill":318,"score_linear":-10752.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98371,"ongoing_decode_tokens":7553,"pending_prefill_tokens":5314,"num_requests":2,"active_p_offloads":0,"cached_blocks":2155,"cache_hit":0,"new_prefill":11070,"score_linear":98371.0,"score_lmetric":32768},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":6656,"new_prefill":4414,"score_linear":99487.0,"score_lmetric":4414},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":6377,"ongoing_decode_tokens":6377,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":0,"new_prefill":11070,"score_linear":6377.0,"score_lmetric":11070},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":11070,"score_linear":361577.0,"score_lmetric":97015},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1768,"cache_hit":0,"new_prefill":11070,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":11070,"score_linear":121811.0,"score_lmetric":70929}],"chosen_score_linear":-10752.0,"chosen_score_lmetric":0,"t_first_token":282146.676167325,"t_first_token_unix":1779713011.6901112,"t_done":282147.378484108,"t_done_unix":1779713012.392427},{"request_id":"1408578:1:1408578:798","session_id":"1408578","input_length":6377,"t_proxy_recv":282145.945951309,"t_decision_unix":1779713010.959893,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":6377,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":6377,"score_linear":262703.0,"score_lmetric":97716},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":109775,"ongoing_decode_tokens":109775,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1715,"cache_hit":0,"new_prefill":6377,"score_linear":109775.0,"score_lmetric":6377},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7553,"ongoing_decode_tokens":0,"pending_prefill_tokens":7553,"num_requests":1,"active_p_offloads":0,"cached_blocks":2155,"cache_hit":0,"new_prefill":6377,"score_linear":7553.0,"score_lmetric":13930},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":6377,"score_linear":106143.0,"score_lmetric":6377},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":0,"new_prefill":6377,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":6377,"score_linear":361577.0,"score_lmetric":73550},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1768,"cache_hit":0,"new_prefill":6377,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":6377,"score_linear":121811.0,"score_lmetric":66236}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282146.367066944,"t_first_token_unix":1779713011.3810108,"t_done":282147.459141956,"t_done_unix":1779713012.473085},{"request_id":"1408430:1:1408430:797","session_id":"1408430","input_length":7553,"t_proxy_recv":282145.530528602,"t_decision_unix":1779713010.5444696,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":7553,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":7553,"score_linear":262703.0,"score_lmetric":102420},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":109775,"ongoing_decode_tokens":0,"pending_prefill_tokens":207,"num_requests":1,"active_p_offloads":0,"cached_blocks":1715,"cache_hit":0,"new_prefill":7553,"score_linear":109775.0,"score_lmetric":7760},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2155,"cache_hit":0,"new_prefill":7553,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":7553,"score_linear":106143.0,"score_lmetric":7553},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":14728,"ongoing_decode_tokens":0,"pending_prefill_tokens":6024,"num_requests":1,"active_p_offloads":0,"cached_blocks":1868,"cache_hit":0,"new_prefill":7553,"score_linear":14728.0,"score_lmetric":13577},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":7553,"score_linear":361577.0,"score_lmetric":79430},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":27786,"ongoing_decode_tokens":27786,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":0,"new_prefill":7553,"score_linear":27786.0,"score_lmetric":7553},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":7553,"score_linear":121811.0,"score_lmetric":67412}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282146.046216843,"t_first_token_unix":1779713011.0601604,"t_done":282149.139701168,"t_done_unix":1779713014.1536443},{"request_id":"1381387:6:1409195:806","session_id":"1381387","input_length":71570,"t_proxy_recv":282148.270373558,"t_decision_unix":1779713013.2843149,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":69632,"affinity_cache_ratio":0.9729216152019002,"affinity_num_requests":0,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":69632,"estimated_new_tokens":1938,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":71570,"score_linear":262703.0,"score_lmetric":358488},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1721,"cache_hit":69632,"new_prefill":1938,"score_linear":-69632.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98371,"ongoing_decode_tokens":7553,"pending_prefill_tokens":5314,"num_requests":2,"active_p_offloads":0,"cached_blocks":2155,"cache_hit":0,"new_prefill":71570,"score_linear":98371.0,"score_lmetric":153768},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":71570,"score_linear":106143.0,"score_lmetric":71570},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1891,"cache_hit":0,"new_prefill":71570,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":71570,"score_linear":361577.0,"score_lmetric":399515},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":104923,"ongoing_decode_tokens":1801,"pending_prefill_tokens":12498,"num_requests":3,"active_p_offloads":0,"cached_blocks":1768,"cache_hit":512,"new_prefill":71058,"score_linear":104411.0,"score_lmetric":250668},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":71570,"score_linear":121811.0,"score_lmetric":131429}],"chosen_score_linear":-69632.0,"chosen_score_lmetric":0,"t_first_token":282149.305620065,"t_first_token_unix":1779713014.3195639,"t_done":282150.243905975,"t_done_unix":1779713015.2578495},{"request_id":"1313181:7:1327489:383","session_id":"1313181","input_length":121811,"t_proxy_recv":282115.284253367,"t_decision_unix":1779712980.298195,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":61952,"affinity_cache_ratio":0.5085911781366215,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":61952,"estimated_new_tokens":59859,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":121811,"score_linear":101803.0,"score_lmetric":150398},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":0,"new_prefill":121811,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2081,"cache_hit":0,"new_prefill":121811,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":169401,"ongoing_decode_tokens":106143,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":121811,"score_linear":169401.0,"score_lmetric":267738},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1769,"cache_hit":0,"new_prefill":121811,"score_linear":33321.0,"score_lmetric":121811},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":12800,"new_prefill":109011,"score_linear":111464.0,"score_lmetric":409206},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":121811,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":61952,"new_prefill":59859,"score_linear":-61952.0,"score_lmetric":0}],"chosen_score_linear":-61952.0,"chosen_score_lmetric":0,"t_first_token":282149.010951604,"t_first_token_unix":1779713014.0248954,"t_done":282150.272058984,"t_done_unix":1779713015.2860026},{"request_id":"1373577:4:1408642:799","session_id":"1373577","input_length":90818,"t_proxy_recv":282146.165739557,"t_decision_unix":1779713011.179681,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":85504,"affinity_cache_ratio":0.9414873703450858,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":85504,"estimated_new_tokens":5314,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":90818,"score_linear":262703.0,"score_lmetric":435480},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1715,"cache_hit":0,"new_prefill":90818,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7553,"ongoing_decode_tokens":7553,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2155,"cache_hit":85504,"new_prefill":5314,"score_linear":-77951.0,"score_lmetric":5314},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":90818,"score_linear":106143.0,"score_lmetric":90818},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":6377,"ongoing_decode_tokens":0,"pending_prefill_tokens":6377,"num_requests":1,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":0,"new_prefill":90818,"score_linear":6377.0,"score_lmetric":97195},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":90818,"score_linear":361577.0,"score_lmetric":495755},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1768,"cache_hit":2560,"new_prefill":88258,"score_linear":-2560.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":90818,"score_linear":121811.0,"score_lmetric":150677}],"chosen_score_linear":-77951.0,"chosen_score_lmetric":5314,"t_first_token":282149.085240854,"t_first_token_unix":1779713014.099187,"t_done":282150.956528207,"t_done_unix":1779713015.970472},{"request_id":"1406966:3:1410097:808","session_id":"1406966","input_length":7893,"t_proxy_recv":282151.379818249,"t_decision_unix":1779713016.3937597,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":4608,"affinity_cache_ratio":0.5838084378563284,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":3285,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":7893,"score_linear":262703.0,"score_lmetric":103780},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1725,"cache_hit":0,"new_prefill":7893,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2179,"cache_hit":4608,"new_prefill":3285,"score_linear":-4608.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":7893,"score_linear":106143.0,"score_lmetric":7893},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1891,"cache_hit":0,"new_prefill":7893,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":7893,"score_linear":361577.0,"score_lmetric":81130},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":104923,"ongoing_decode_tokens":75193,"pending_prefill_tokens":2082,"num_requests":3,"active_p_offloads":0,"cached_blocks":1768,"cache_hit":2048,"new_prefill":5845,"score_linear":102875.0,"score_lmetric":23781},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":62018,"ongoing_decode_tokens":62018,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":7893,"score_linear":62018.0,"score_lmetric":7893}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":282151.658755813,"t_first_token_unix":1779713016.6727002,"t_done":282151.857780931,"t_done_unix":1779713016.8717246},{"request_id":"1408779:1:1408779:802","session_id":"1408779","input_length":1801,"t_proxy_recv":282146.633183742,"t_decision_unix":1779713011.6471252,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":1801,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":1801,"score_linear":262703.0,"score_lmetric":79412},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":11070,"ongoing_decode_tokens":0,"pending_prefill_tokens":318,"num_requests":1,"active_p_offloads":0,"cached_blocks":1715,"cache_hit":0,"new_prefill":1801,"score_linear":11070.0,"score_lmetric":2119},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98371,"ongoing_decode_tokens":7553,"pending_prefill_tokens":5314,"num_requests":2,"active_p_offloads":0,"cached_blocks":2155,"cache_hit":0,"new_prefill":1801,"score_linear":98371.0,"score_lmetric":14230},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":1801,"score_linear":106143.0,"score_lmetric":1801},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":6377,"ongoing_decode_tokens":6377,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":0,"new_prefill":1801,"score_linear":6377.0,"score_lmetric":1801},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":1801,"score_linear":361577.0,"score_lmetric":50670},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1768,"cache_hit":0,"new_prefill":1801,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":1801,"score_linear":121811.0,"score_lmetric":61660}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282146.781482553,"t_first_token_unix":1779713011.795426,"t_done":282152.213874131,"t_done_unix":1779713017.2278175},{"request_id":"1407494:2:1410400:809","session_id":"1407494","input_length":4728,"t_proxy_recv":282152.243842237,"t_decision_unix":1779713017.2577837,"policy":"unified","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":4,"affinity_cache_hit":1536,"affinity_cache_ratio":0.3248730964467005,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4728,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":4728,"score_linear":262703.0,"score_lmetric":91120},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1725,"cache_hit":0,"new_prefill":4728,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":5001,"ongoing_decode_tokens":0,"pending_prefill_tokens":5001,"num_requests":1,"active_p_offloads":0,"cached_blocks":2185,"cache_hit":1536,"new_prefill":3192,"score_linear":3465.0,"score_lmetric":8193},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":4728,"score_linear":106143.0,"score_lmetric":4728},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1891,"cache_hit":0,"new_prefill":4728,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":4728,"score_linear":361577.0,"score_lmetric":65305},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":103122,"ongoing_decode_tokens":103122,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":0,"new_prefill":4728,"score_linear":103122.0,"score_lmetric":9456},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":62018,"ongoing_decode_tokens":62018,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":4728,"score_linear":62018.0,"score_lmetric":4728}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282152.54346256,"t_first_token_unix":1779713017.5574057,"t_done":282152.680441463,"t_done_unix":1779713017.6943848},{"request_id":"1408779:2:1410032:807","session_id":"1408779","input_length":5001,"t_proxy_recv":282152.217893282,"t_decision_unix":1779713017.231835,"policy":"unified","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":2,"affinity_cache_hit":1536,"affinity_cache_ratio":0.30713857228554287,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5001,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":5001,"score_linear":262703.0,"score_lmetric":92212},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1725,"cache_hit":0,"new_prefill":5001,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2185,"cache_hit":0,"new_prefill":5001,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":5001,"score_linear":106143.0,"score_lmetric":5001},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1891,"cache_hit":0,"new_prefill":5001,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":5001,"score_linear":361577.0,"score_lmetric":66670},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":103122,"ongoing_decode_tokens":103122,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":1536,"new_prefill":3465,"score_linear":101586.0,"score_lmetric":6930},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":62018,"ongoing_decode_tokens":62018,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":5001,"score_linear":62018.0,"score_lmetric":5001}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282152.53328658,"t_first_token_unix":1779713017.54723,"t_done":282152.682459298,"t_done_unix":1779713017.696402},{"request_id":"1408889:2:1410453:810","session_id":"1408889","input_length":4825,"t_proxy_recv":282152.382423396,"t_decision_unix":1779713017.3963645,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":3072,"affinity_cache_ratio":0.6366839378238341,"affinity_num_requests":0,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":3072,"estimated_new_tokens":1753,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":4825,"score_linear":262703.0,"score_lmetric":91508},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1725,"cache_hit":3072,"new_prefill":1753,"score_linear":-3072.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":5001,"ongoing_decode_tokens":0,"pending_prefill_tokens":5001,"num_requests":1,"active_p_offloads":0,"cached_blocks":2185,"cache_hit":0,"new_prefill":4825,"score_linear":5001.0,"score_lmetric":9826},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":4825,"score_linear":106143.0,"score_lmetric":4825},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":4728,"ongoing_decode_tokens":0,"pending_prefill_tokens":4728,"num_requests":1,"active_p_offloads":0,"cached_blocks":1891,"cache_hit":0,"new_prefill":4825,"score_linear":4728.0,"score_lmetric":9553},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":4825,"score_linear":361577.0,"score_lmetric":65790},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":103122,"ongoing_decode_tokens":103122,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":0,"new_prefill":4825,"score_linear":103122.0,"score_lmetric":9650},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":62018,"ongoing_decode_tokens":62018,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":4825,"score_linear":62018.0,"score_lmetric":4825}],"chosen_score_linear":-3072.0,"chosen_score_lmetric":0,"t_first_token":282152.526881779,"t_first_token_unix":1779713017.5408256,"t_done":282152.811284231,"t_done_unix":1779713017.8252273},{"request_id":"1313181:7:1382851:662","session_id":"1313181","input_length":62018,"t_proxy_recv":282150.282728009,"t_decision_unix":1779713015.2966695,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":61952,"affinity_cache_ratio":0.9989357928343384,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":61952,"estimated_new_tokens":66,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":62018,"score_linear":262703.0,"score_lmetric":320280},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1725,"cache_hit":0,"new_prefill":62018,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":90818,"ongoing_decode_tokens":90818,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2169,"cache_hit":0,"new_prefill":62018,"score_linear":90818.0,"score_lmetric":62018},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":62018,"score_linear":106143.0,"score_lmetric":62018},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1891,"cache_hit":0,"new_prefill":62018,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":12800,"new_prefill":49218,"score_linear":348777.0,"score_lmetric":287755},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":104923,"ongoing_decode_tokens":1801,"pending_prefill_tokens":12498,"num_requests":3,"active_p_offloads":0,"cached_blocks":1768,"cache_hit":0,"new_prefill":62018,"score_linear":104923.0,"score_lmetric":223548},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":61952,"new_prefill":66,"score_linear":-61952.0,"score_lmetric":0}],"chosen_score_linear":-61952.0,"chosen_score_lmetric":0,"t_first_token":282150.372911469,"t_first_token_unix":1779713015.3868551,"t_done":282153.65241968,"t_done_unix":1779713018.6663635},{"request_id":"1399948:6:1410696:812","session_id":"1399948","input_length":11381,"t_proxy_recv":282153.133599063,"t_decision_unix":1779713018.1475403,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":8192,"affinity_cache_ratio":0.7197961514805378,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":8192,"estimated_new_tokens":3189,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":11381,"score_linear":262703.0,"score_lmetric":117732},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":0,"new_prefill":11381,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2194,"cache_hit":0,"new_prefill":11381,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":11381,"score_linear":106143.0,"score_lmetric":11381},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":11381,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":11381,"score_linear":361577.0,"score_lmetric":98570},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":103122,"ongoing_decode_tokens":103122,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":8192,"new_prefill":3189,"score_linear":94930.0,"score_lmetric":6378},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":62018,"ongoing_decode_tokens":62018,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":11381,"score_linear":62018.0,"score_lmetric":11381}],"chosen_score_linear":94930.0,"chosen_score_lmetric":6378,"t_first_token":282153.477545445,"t_first_token_unix":1779713018.491489,"t_done":282153.688359504,"t_done_unix":1779713018.702303},{"request_id":"1340290:8:1409129:805","session_id":"1340290","input_length":29730,"t_proxy_recv":282148.006247668,"t_decision_unix":1779713013.020189,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":27648,"affinity_cache_ratio":0.9299697275479314,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":27648,"estimated_new_tokens":2082,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":29730,"score_linear":262703.0,"score_lmetric":191128},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1721,"cache_hit":0,"new_prefill":29730,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98371,"ongoing_decode_tokens":7553,"pending_prefill_tokens":5314,"num_requests":2,"active_p_offloads":0,"cached_blocks":2155,"cache_hit":0,"new_prefill":29730,"score_linear":98371.0,"score_lmetric":70088},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":29730,"score_linear":106143.0,"score_lmetric":29730},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1891,"cache_hit":0,"new_prefill":29730,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":29730,"score_linear":361577.0,"score_lmetric":190315},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":75193,"ongoing_decode_tokens":1801,"pending_prefill_tokens":10416,"num_requests":2,"active_p_offloads":0,"cached_blocks":1768,"cache_hit":27648,"new_prefill":2082,"score_linear":47545.0,"score_lmetric":24996},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":29730,"score_linear":121811.0,"score_lmetric":89589}],"chosen_score_linear":47545.0,"chosen_score_lmetric":24996,"t_first_token":282151.711811272,"t_first_token_unix":1779713016.725755,"t_done":282154.253482775,"t_done_unix":1779713019.2674263},{"request_id":"1340290:9:1410595:811","session_id":"1340290","input_length":29958,"t_proxy_recv":282154.260815638,"t_decision_unix":1779713019.2747567,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":29696,"affinity_cache_ratio":0.9912544228586688,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":29696,"estimated_new_tokens":262,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":29958,"score_linear":262703.0,"score_lmetric":192040},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":0,"new_prefill":29958,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2194,"cache_hit":0,"new_prefill":29958,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":29958,"score_linear":106143.0,"score_lmetric":29958},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":29958,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":29958,"score_linear":361577.0,"score_lmetric":191455},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73392,"ongoing_decode_tokens":73392,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":29696,"new_prefill":262,"score_linear":43696.0,"score_lmetric":262},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":62138,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":29958,"score_linear":124090.0,"score_lmetric":92096}],"chosen_score_linear":43696.0,"chosen_score_lmetric":262,"t_first_token":282154.348565111,"t_first_token_unix":1779713019.3625088,"t_done":282154.868540532,"t_done_unix":1779713019.8824842},{"request_id":"1408779:3:1411249:815","session_id":"1408779","input_length":7110,"t_proxy_recv":282155.160545068,"t_decision_unix":1779713020.1744866,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":4608,"affinity_cache_ratio":0.6481012658227848,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":2502,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":7110,"score_linear":262703.0,"score_lmetric":100648},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117746,"ongoing_decode_tokens":0,"pending_prefill_tokens":8178,"num_requests":1,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":0,"new_prefill":7110,"score_linear":117746.0,"score_lmetric":15288},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2194,"cache_hit":4608,"new_prefill":2502,"score_linear":-4608.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":7110,"score_linear":106143.0,"score_lmetric":7110},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":7110,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":7110,"score_linear":361577.0,"score_lmetric":77215},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73392,"ongoing_decode_tokens":73392,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":1536,"new_prefill":5574,"score_linear":71856.0,"score_lmetric":5574},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":62138,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":7110,"score_linear":124090.0,"score_lmetric":69248}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":282155.359156249,"t_first_token_unix":1779713020.3730993,"t_done":282155.515656575,"t_done_unix":1779713020.5295997},{"request_id":"1364090:6:1408777:801","session_id":"1364090","input_length":73392,"t_proxy_recv":282146.63861653,"t_decision_unix":1779713011.652558,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":62976,"affinity_cache_ratio":0.8580771746239372,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":62976,"estimated_new_tokens":10416,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":7680,"new_prefill":65712,"score_linear":255023.0,"score_lmetric":335056},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":11070,"ongoing_decode_tokens":0,"pending_prefill_tokens":318,"num_requests":1,"active_p_offloads":0,"cached_blocks":1715,"cache_hit":4096,"new_prefill":69296,"score_linear":6974.0,"score_lmetric":69614},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98371,"ongoing_decode_tokens":7553,"pending_prefill_tokens":5314,"num_requests":2,"active_p_offloads":0,"cached_blocks":2155,"cache_hit":0,"new_prefill":73392,"score_linear":98371.0,"score_lmetric":157412},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":7680,"new_prefill":65712,"score_linear":98463.0,"score_lmetric":65712},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":6377,"ongoing_decode_tokens":6377,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":0,"new_prefill":73392,"score_linear":6377.0,"score_lmetric":73392},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":73392,"score_linear":361577.0,"score_lmetric":408625},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":1801,"ongoing_decode_tokens":0,"pending_prefill_tokens":1801,"num_requests":1,"active_p_offloads":0,"cached_blocks":1768,"cache_hit":62976,"new_prefill":10416,"score_linear":-61175.0,"score_lmetric":12217},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":7680,"new_prefill":65712,"score_linear":114131.0,"score_lmetric":125571}],"chosen_score_linear":-61175.0,"chosen_score_lmetric":12217,"t_first_token":282151.274860511,"t_first_token_unix":1779713016.2888093,"t_done":282156.580905322,"t_done_unix":1779713021.5948484},{"request_id":"1340290:10:1411397:816","session_id":"1340290","input_length":30154,"t_proxy_recv":282155.788486679,"t_decision_unix":1779713020.8024282,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":29696,"affinity_cache_ratio":0.9848113019831531,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":29696,"estimated_new_tokens":458,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":30154,"score_linear":262703.0,"score_lmetric":192824},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117746,"ongoing_decode_tokens":0,"pending_prefill_tokens":8178,"num_requests":1,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":0,"new_prefill":30154,"score_linear":117746.0,"score_lmetric":38332},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2198,"cache_hit":0,"new_prefill":30154,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":30154,"score_linear":106143.0,"score_lmetric":30154},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":30154,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":30154,"score_linear":361577.0,"score_lmetric":192435},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73392,"ongoing_decode_tokens":73392,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":29696,"new_prefill":458,"score_linear":43696.0,"score_lmetric":458},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":62138,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":30154,"score_linear":124090.0,"score_lmetric":92292}],"chosen_score_linear":43696.0,"chosen_score_lmetric":458,"t_first_token":282155.908742672,"t_first_token_unix":1779713020.9226859,"t_done":282158.037339092,"t_done_unix":1779713023.0512822},{"request_id":"1370129:6:1394177:722","session_id":"1370129","input_length":106143,"t_proxy_recv":282095.747536001,"t_decision_unix":1779712960.7614775,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":88064,"affinity_cache_ratio":0.8296731767521174,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":88064,"estimated_new_tokens":18079,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":7680,"new_prefill":98463,"score_linear":94123.0,"score_lmetric":127050},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1595,"cache_hit":4096,"new_prefill":102047,"score_linear":-4096.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":4675,"ongoing_decode_tokens":0,"pending_prefill_tokens":4675,"num_requests":1,"active_p_offloads":0,"cached_blocks":2012,"cache_hit":0,"new_prefill":106143,"score_linear":4675.0,"score_lmetric":110818},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":88064,"new_prefill":18079,"score_linear":-72781.0,"score_lmetric":18079},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":0,"pending_prefill_tokens":5161,"num_requests":1,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":106143,"score_linear":33321.0,"score_lmetric":111304},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":0,"new_prefill":106143,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":48944,"ongoing_decode_tokens":48944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1575,"cache_hit":7680,"new_prefill":98463,"score_linear":41264.0,"score_lmetric":98463},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":7680,"new_prefill":98463,"score_linear":237633.0,"score_lmetric":431046}],"chosen_score_linear":-72781.0,"chosen_score_lmetric":18079,"t_first_token":282106.654990348,"t_first_token_unix":1779712971.6689343,"t_done":282159.843567677,"t_done_unix":1779713024.8575108},{"request_id":"1366128:25:1411126:813","session_id":"1366128","input_length":117746,"t_proxy_recv":282154.759473342,"t_decision_unix":1779713019.7734146,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":109568,"affinity_cache_ratio":0.9305454113090891,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":109568,"estimated_new_tokens":8178,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":117746,"score_linear":262703.0,"score_lmetric":543192},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":109568,"new_prefill":8178,"score_linear":-109568.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2194,"cache_hit":0,"new_prefill":117746,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":117746,"score_linear":106143.0,"score_lmetric":117746},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":117746,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":117746,"score_linear":361577.0,"score_lmetric":630395},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":103350,"ongoing_decode_tokens":103350,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":0,"new_prefill":117746,"score_linear":103350.0,"score_lmetric":235492},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":62138,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":117746,"score_linear":124090.0,"score_lmetric":179884}],"chosen_score_linear":-109568.0,"chosen_score_lmetric":0,"t_first_token":282160.327524841,"t_first_token_unix":1779713025.3414705,"t_done":282161.531219394,"t_done_unix":1779713026.545163},{"request_id":"1407494:3:1413526:825","session_id":"1407494","input_length":9801,"t_proxy_recv":282163.195412181,"t_decision_unix":1779713028.2093534,"policy":"unified","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":2,"affinity_cache_hit":4608,"affinity_cache_ratio":0.4701561065197429,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":0,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":8265,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":9801,"score_linear":262703.0,"score_lmetric":111412},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":200597,"ongoing_decode_tokens":72608,"pending_prefill_tokens":10229,"num_requests":2,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":9801,"score_linear":200597.0,"score_lmetric":40060},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2198,"cache_hit":1536,"new_prefill":8265,"score_linear":-1536.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":40273,"ongoing_decode_tokens":0,"pending_prefill_tokens":1873,"num_requests":1,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":9801,"score_linear":40273.0,"score_lmetric":11674},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":4608,"new_prefill":5193,"score_linear":102684.0,"score_lmetric":19301},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":9801,"score_linear":361577.0,"score_lmetric":90670},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":132752,"ongoing_decode_tokens":0,"pending_prefill_tokens":17552,"num_requests":3,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":9801,"score_linear":132752.0,"score_lmetric":82059},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":62138,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":9801,"score_linear":124090.0,"score_lmetric":71939}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":282163.855130029,"t_first_token_unix":1779713028.8690734,"t_done":282164.191060533,"t_done_unix":1779713029.2050045},{"request_id":"1408889:3:1414177:827","session_id":"1408889","input_length":10633,"t_proxy_recv":282165.326282837,"t_decision_unix":1779713030.3402238,"policy":"unified","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":2,"affinity_cache_hit":4608,"affinity_cache_ratio":0.43336781717295214,"affinity_num_requests":2,"avg_num_requests":2.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":10633,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":10633,"score_linear":262703.0,"score_lmetric":114740},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":200597,"ongoing_decode_tokens":72608,"pending_prefill_tokens":10229,"num_requests":2,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":4608,"new_prefill":6025,"score_linear":195989.0,"score_lmetric":32508},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2214,"cache_hit":0,"new_prefill":10633,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":62665,"ongoing_decode_tokens":40273,"pending_prefill_tokens":22392,"num_requests":2,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":10633,"score_linear":62665.0,"score_lmetric":66050},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":10633,"score_linear":107292.0,"score_lmetric":24741},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":10633,"score_linear":361577.0,"score_lmetric":94830},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":132752,"ongoing_decode_tokens":0,"pending_prefill_tokens":17552,"num_requests":3,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":10633,"score_linear":132752.0,"score_lmetric":84555},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":62138,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":10633,"score_linear":124090.0,"score_lmetric":72771}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282166.1561107,"t_first_token_unix":1779713031.1700542,"t_done":282167.176087308,"t_done_unix":1779713032.1900308},{"request_id":"1407995:2:1413725:826","session_id":"1407995","input_length":22392,"t_proxy_recv":282163.839712068,"t_decision_unix":1779713028.8536532,"policy":"unified","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":3,"affinity_cache_hit":2560,"affinity_cache_ratio":0.1143265451947124,"affinity_num_requests":1,"avg_num_requests":2.25,"fallback_score":24265,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":22392,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":22392,"score_linear":262703.0,"score_lmetric":161776},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":200597,"ongoing_decode_tokens":72608,"pending_prefill_tokens":10229,"num_requests":2,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":22392,"score_linear":200597.0,"score_lmetric":65242},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":9801,"ongoing_decode_tokens":0,"pending_prefill_tokens":8265,"num_requests":1,"active_p_offloads":0,"cached_blocks":2198,"cache_hit":0,"new_prefill":22392,"score_linear":9801.0,"score_lmetric":30657},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":40273,"ongoing_decode_tokens":0,"pending_prefill_tokens":1873,"num_requests":1,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":22392,"score_linear":40273.0,"score_lmetric":24265},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":2560,"new_prefill":19832,"score_linear":104732.0,"score_lmetric":33940},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":22392,"score_linear":361577.0,"score_lmetric":153625},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":132752,"ongoing_decode_tokens":0,"pending_prefill_tokens":17552,"num_requests":3,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":22392,"score_linear":132752.0,"score_lmetric":119832},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":62138,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":22392,"score_linear":124090.0,"score_lmetric":84530}],"chosen_score_linear":40273.0,"chosen_score_lmetric":24265,"t_first_token":282167.286420858,"t_first_token_unix":1779713032.3003647,"t_done":282167.934535721,"t_done_unix":1779713032.9484794},{"request_id":"1277428:8:1412675:821","session_id":"1277428","input_length":40273,"t_proxy_recv":282160.118482229,"t_decision_unix":1779713025.132423,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":38400,"affinity_cache_ratio":0.9534924142725896,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":38400,"estimated_new_tokens":1873,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":40273,"score_linear":262703.0,"score_lmetric":233300},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":190354,"ongoing_decode_tokens":0,"pending_prefill_tokens":9106,"num_requests":2,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":4608,"new_prefill":35665,"score_linear":185746.0,"score_lmetric":89542},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2198,"cache_hit":0,"new_prefill":40273,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":38400,"new_prefill":1873,"score_linear":-38400.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":40273,"score_linear":107292.0,"score_lmetric":54381},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":40273,"score_linear":361577.0,"score_lmetric":243030},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":86193,"ongoing_decode_tokens":0,"pending_prefill_tokens":12465,"num_requests":1,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":40273,"score_linear":86193.0,"score_lmetric":52738},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":62138,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":40273,"score_linear":124090.0,"score_lmetric":102411}],"chosen_score_linear":-38400.0,"chosen_score_lmetric":0,"t_first_token":282164.892440831,"t_first_token_unix":1779713029.9063845,"t_done":282168.765912698,"t_done_unix":1779713033.7798562},{"request_id":"1399948:7:1413205:824","session_id":"1399948","input_length":11537,"t_proxy_recv":282162.008934395,"t_decision_unix":1779713027.0228755,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":11264,"affinity_cache_ratio":0.9763370026870071,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":273,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":11537,"score_linear":262703.0,"score_lmetric":118356},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":200597,"ongoing_decode_tokens":72608,"pending_prefill_tokens":10229,"num_requests":2,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":11537,"score_linear":200597.0,"score_lmetric":43532},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2198,"cache_hit":0,"new_prefill":11537,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":40273,"ongoing_decode_tokens":0,"pending_prefill_tokens":1873,"num_requests":1,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":11537,"score_linear":40273.0,"score_lmetric":13410},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":11537,"score_linear":107292.0,"score_lmetric":25645},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":11537,"score_linear":361577.0,"score_lmetric":99350},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121215,"ongoing_decode_tokens":0,"pending_prefill_tokens":17279,"num_requests":2,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":11264,"new_prefill":273,"score_linear":109951.0,"score_lmetric":35104},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":62138,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":11537,"score_linear":124090.0,"score_lmetric":73675}],"chosen_score_linear":109951.0,"chosen_score_lmetric":35104,"t_first_token":282167.35705046,"t_first_token_unix":1779713032.3709936,"t_done":282169.871623436,"t_done_unix":1779713034.8855665},{"request_id":"1340290:11:1413050:822","session_id":"1340290","input_length":35022,"t_proxy_recv":282161.407324069,"t_decision_unix":1779713026.4212654,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":30208,"affinity_cache_ratio":0.8625435440580207,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":30208,"estimated_new_tokens":4814,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":35022,"score_linear":262703.0,"score_lmetric":212296},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":190354,"ongoing_decode_tokens":190354,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":0,"new_prefill":35022,"score_linear":190354.0,"score_lmetric":70044},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2198,"cache_hit":0,"new_prefill":35022,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":40273,"ongoing_decode_tokens":0,"pending_prefill_tokens":1873,"num_requests":1,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":35022,"score_linear":40273.0,"score_lmetric":36895},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":35022,"score_linear":107292.0,"score_lmetric":49130},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":35022,"score_linear":361577.0,"score_lmetric":216775},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":86193,"ongoing_decode_tokens":0,"pending_prefill_tokens":12465,"num_requests":1,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":30208,"new_prefill":4814,"score_linear":55985.0,"score_lmetric":17279},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":62138,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":35022,"score_linear":124090.0,"score_lmetric":97160}],"chosen_score_linear":55985.0,"chosen_score_lmetric":17279,"t_first_token":282167.356332173,"t_first_token_unix":1779713032.3702762,"t_done":282173.058633039,"t_done_unix":1779713038.0725765},{"request_id":"1366128:26:1413108:823","session_id":"1366128","input_length":127989,"t_proxy_recv":282161.63858303,"t_decision_unix":1779713026.6525245,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":117760,"affinity_cache_ratio":0.9200790692950176,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":117760,"estimated_new_tokens":10229,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":127989,"score_linear":262703.0,"score_lmetric":584164},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":117760,"new_prefill":10229,"score_linear":-45152.0,"score_lmetric":10229},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2198,"cache_hit":0,"new_prefill":127989,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":40273,"ongoing_decode_tokens":0,"pending_prefill_tokens":1873,"num_requests":1,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":127989,"score_linear":40273.0,"score_lmetric":129862},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":127989,"score_linear":107292.0,"score_lmetric":142097},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":127989,"score_linear":361577.0,"score_lmetric":681610},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121215,"ongoing_decode_tokens":0,"pending_prefill_tokens":17279,"num_requests":2,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":127989,"score_linear":121215.0,"score_lmetric":290536},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":62138,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":127989,"score_linear":124090.0,"score_lmetric":190127}],"chosen_score_linear":-45152.0,"chosen_score_lmetric":10229,"t_first_token":282169.426805012,"t_first_token_unix":1779713034.4407485,"t_done":282173.328120968,"t_done_unix":1779713038.3420641},{"request_id":"1364090:7:1412644:820","session_id":"1364090","input_length":86193,"t_proxy_recv":282160.020516022,"t_decision_unix":1779713025.0344574,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":73728,"affinity_cache_ratio":0.8553826876892555,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":73728,"estimated_new_tokens":12465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":7680,"new_prefill":78513,"score_linear":255023.0,"score_lmetric":386260},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":190354,"ongoing_decode_tokens":0,"pending_prefill_tokens":9106,"num_requests":2,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":4096,"new_prefill":82097,"score_linear":186258.0,"score_lmetric":182406},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2198,"cache_hit":0,"new_prefill":86193,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":7680,"new_prefill":78513,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":86193,"score_linear":107292.0,"score_lmetric":100301},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":86193,"score_linear":361577.0,"score_lmetric":472630},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":73728,"new_prefill":12465,"score_linear":-73728.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":62138,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":7680,"new_prefill":78513,"score_linear":116410.0,"score_lmetric":140651}],"chosen_score_linear":-73728.0,"chosen_score_lmetric":0,"t_first_token":282166.306245467,"t_first_token_unix":1779713031.320189,"t_done":282176.467424846,"t_done_unix":1779713041.4813685},{"request_id":"1405155:2:1414214:828","session_id":"1405155","input_length":40330,"t_proxy_recv":282165.476238127,"t_decision_unix":1779713030.490179,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":33280,"affinity_cache_ratio":0.825192164641706,"affinity_num_requests":3,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"cache_hit":33280,"estimated_new_tokens":7050,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":40330,"score_linear":262703.0,"score_lmetric":233528},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":200597,"ongoing_decode_tokens":72608,"pending_prefill_tokens":10229,"num_requests":2,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":512,"new_prefill":39818,"score_linear":200085.0,"score_lmetric":100094},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10633,"ongoing_decode_tokens":0,"pending_prefill_tokens":10633,"num_requests":1,"active_p_offloads":0,"cached_blocks":2214,"cache_hit":0,"new_prefill":40330,"score_linear":10633.0,"score_lmetric":50963},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":62665,"ongoing_decode_tokens":40273,"pending_prefill_tokens":22392,"num_requests":2,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":40330,"score_linear":62665.0,"score_lmetric":125444},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":40330,"score_linear":107292.0,"score_lmetric":54438},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":40330,"score_linear":361577.0,"score_lmetric":243315},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":132752,"ongoing_decode_tokens":0,"pending_prefill_tokens":17552,"num_requests":3,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":33280,"new_prefill":7050,"score_linear":99472.0,"score_lmetric":73806},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":62138,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":40330,"score_linear":124090.0,"score_lmetric":102468}],"chosen_score_linear":99472.0,"chosen_score_lmetric":73806,"t_first_token":282169.06078236,"t_first_token_unix":1779713034.0747259,"t_done":282176.610518408,"t_done_unix":1779713041.6244617},{"request_id":"1366128:27:1417788:831","session_id":"1366128","input_length":128433,"t_proxy_recv":282178.225590834,"t_decision_unix":1779713043.239532,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":128000,"affinity_cache_ratio":0.9966285923399749,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":128000,"estimated_new_tokens":433,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":235027,"pending_prefill_tokens":5660,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":128433,"score_linear":262703.0,"score_lmetric":536372},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":128000,"new_prefill":433,"score_linear":-55392.0,"score_lmetric":433},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2235,"cache_hit":0,"new_prefill":128433,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":0,"new_prefill":128433,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":128433,"score_linear":107292.0,"score_lmetric":142541},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":128433,"score_linear":361577.0,"score_lmetric":683830},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":96849,"ongoing_decode_tokens":0,"pending_prefill_tokens":10321,"num_requests":1,"active_p_offloads":0,"cached_blocks":1851,"cache_hit":0,"new_prefill":128433,"score_linear":96849.0,"score_lmetric":138754},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":167853,"ongoing_decode_tokens":0,"pending_prefill_tokens":64941,"num_requests":2,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":128433,"score_linear":167853.0,"score_lmetric":386748}],"chosen_score_linear":-55392.0,"chosen_score_lmetric":433,"t_first_token":282178.626397859,"t_first_token_unix":1779713043.6403415,"t_done":282180.022842106,"t_done_unix":1779713045.036786},{"request_id":"1400379:4:1405483:779","session_id":"1400379","input_length":19966,"t_proxy_recv":282135.015665753,"t_decision_unix":1779713000.029607,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":10752,"affinity_cache_ratio":0.5385154763097265,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":10752,"estimated_new_tokens":9214,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":215061,"ongoing_decode_tokens":101803,"pending_prefill_tokens":3178,"num_requests":2,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":10752,"new_prefill":9214,"score_linear":204309.0,"score_lmetric":24784},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":69508,"ongoing_decode_tokens":0,"pending_prefill_tokens":1412,"num_requests":1,"active_p_offloads":0,"cached_blocks":1689,"cache_hit":3584,"new_prefill":16382,"score_linear":65924.0,"score_lmetric":17794},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":145650,"ongoing_decode_tokens":21009,"pending_prefill_tokens":5345,"num_requests":3,"active_p_offloads":0,"cached_blocks":2107,"cache_hit":0,"new_prefill":19966,"score_linear":145650.0,"score_lmetric":75933},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":117646,"ongoing_decode_tokens":106143,"pending_prefill_tokens":11503,"num_requests":2,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":19966,"score_linear":117646.0,"score_lmetric":62938},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":19966,"score_linear":21684.0,"score_lmetric":19966},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168412,"ongoing_decode_tokens":0,"pending_prefill_tokens":98268,"num_requests":3,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":19966,"score_linear":168412.0,"score_lmetric":354702},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":39093,"ongoing_decode_tokens":0,"pending_prefill_tokens":34485,"num_requests":2,"active_p_offloads":0,"cached_blocks":1691,"cache_hit":0,"new_prefill":19966,"score_linear":39093.0,"score_lmetric":108902},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":19966,"score_linear":121811.0,"score_lmetric":79825}],"chosen_score_linear":204309.0,"chosen_score_lmetric":24784,"t_first_token":282177.030007028,"t_first_token_unix":1779713042.043951,"t_done":282181.468655012,"t_done_unix":1779713046.4825983},{"request_id":"1397515:1:1397515:736","session_id":"1397515","input_length":93040,"t_proxy_recv":282107.253064709,"t_decision_unix":1779712972.2670057,"policy":"unified","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":93040,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":93040,"score_linear":101803.0,"score_lmetric":121627},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":98638,"ongoing_decode_tokens":0,"pending_prefill_tokens":13646,"num_requests":2,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":512,"new_prefill":92528,"score_linear":98126.0,"score_lmetric":212348},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":0,"new_prefill":93040,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":93040,"score_linear":106143.0,"score_lmetric":93040},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":50368,"ongoing_decode_tokens":50368,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":93040,"score_linear":50368.0,"score_lmetric":186080},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":93040,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":93040,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":207071,"ongoing_decode_tokens":207071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":93040,"score_linear":207071.0,"score_lmetric":186080}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282136.496487436,"t_first_token_unix":1779713001.5104308,"t_done":282182.961868354,"t_done_unix":1779713047.9758122},{"request_id":"1400379:5:1406380:784","session_id":"1400379","input_length":22009,"t_proxy_recv":282181.474904523,"t_decision_unix":1779713046.4888458,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":19968,"affinity_cache_ratio":0.9072652096869462,"affinity_num_requests":3,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":2041,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":242737,"ongoing_decode_tokens":242737,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":19968,"new_prefill":2041,"score_linear":222769.0,"score_lmetric":6123},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1765,"cache_hit":3584,"new_prefill":18425,"score_linear":69024.0,"score_lmetric":18425},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37398,"ongoing_decode_tokens":0,"pending_prefill_tokens":37398,"num_requests":1,"active_p_offloads":0,"cached_blocks":2235,"cache_hit":0,"new_prefill":22009,"score_linear":37398.0,"score_lmetric":59407},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":0,"new_prefill":22009,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":22009,"score_linear":107292.0,"score_lmetric":36117},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":22009,"score_linear":361577.0,"score_lmetric":151710},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":96849,"ongoing_decode_tokens":0,"pending_prefill_tokens":10321,"num_requests":1,"active_p_offloads":0,"cached_blocks":1851,"cache_hit":0,"new_prefill":22009,"score_linear":96849.0,"score_lmetric":32330},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":167853,"ongoing_decode_tokens":0,"pending_prefill_tokens":64941,"num_requests":2,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":22009,"score_linear":167853.0,"score_lmetric":173900}],"chosen_score_linear":222769.0,"chosen_score_lmetric":6123,"t_first_token":282181.920243321,"t_first_token_unix":1779713046.934187,"t_done":282183.240299439,"t_done_unix":1779713048.2542427},{"request_id":"1362265:8:1419710:834","session_id":"1362265","input_length":11132,"t_proxy_recv":282184.910015424,"t_decision_unix":1779713049.9239566,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9658641753503413,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":10752,"estimated_new_tokens":380,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":242737,"ongoing_decode_tokens":242737,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1540,"cache_hit":6656,"new_prefill":4476,"score_linear":236081.0,"score_lmetric":13428},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1765,"cache_hit":10752,"new_prefill":380,"score_linear":61856.0,"score_lmetric":380},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37398,"ongoing_decode_tokens":37398,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2235,"cache_hit":0,"new_prefill":11132,"score_linear":37398.0,"score_lmetric":11132},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32179,"ongoing_decode_tokens":0,"pending_prefill_tokens":25523,"num_requests":1,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":6656,"new_prefill":4476,"score_linear":25523.0,"score_lmetric":29999},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":11132,"score_linear":107292.0,"score_lmetric":25240},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":268537,"ongoing_decode_tokens":75372,"pending_prefill_tokens":8333,"num_requests":4,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":0,"new_prefill":11132,"score_linear":268537.0,"score_lmetric":77860},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":96849,"ongoing_decode_tokens":96849,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1851,"cache_hit":0,"new_prefill":11132,"score_linear":96849.0,"score_lmetric":11132},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":167853,"ongoing_decode_tokens":0,"pending_prefill_tokens":64941,"num_requests":2,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":11132,"score_linear":167853.0,"score_lmetric":152146}],"chosen_score_linear":61856.0,"chosen_score_lmetric":380,"t_first_token":282184.959685695,"t_first_token_unix":1779713049.9736292,"t_done":282185.424917765,"t_done_unix":1779713050.4388611},{"request_id":"1286804:5:1407989:792","session_id":"1286804","input_length":27676,"t_proxy_recv":282143.970836361,"t_decision_unix":1779713008.9847772,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":22016,"affinity_cache_ratio":0.795490677843619,"affinity_num_requests":3,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":22016,"estimated_new_tokens":5660,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":235027,"ongoing_decode_tokens":101803,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":22016,"new_prefill":5660,"score_linear":213011.0,"score_lmetric":54156},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":179082,"ongoing_decode_tokens":69508,"pending_prefill_tokens":11782,"num_requests":2,"active_p_offloads":0,"cached_blocks":1689,"cache_hit":0,"new_prefill":27676,"score_linear":179082.0,"score_lmetric":78916},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2155,"cache_hit":0,"new_prefill":27676,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":27676,"score_linear":106143.0,"score_lmetric":27676},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1863,"cache_hit":2560,"new_prefill":25116,"score_linear":-2560.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":124264,"pending_prefill_tokens":11009,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":27676,"score_linear":361577.0,"score_lmetric":193425},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32944,"ongoing_decode_tokens":32944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1702,"cache_hit":0,"new_prefill":27676,"score_linear":32944.0,"score_lmetric":27676},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":27676,"score_linear":121811.0,"score_lmetric":87535}],"chosen_score_linear":213011.0,"chosen_score_lmetric":54156,"t_first_token":282180.553116994,"t_first_token_unix":1779713045.5670612,"t_done":282185.778813908,"t_done_unix":1779713050.7927575},{"request_id":"1364090:8:1417347:830","session_id":"1364090","input_length":96849,"t_proxy_recv":282176.679462723,"t_decision_unix":1779713041.693404,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":86528,"affinity_cache_ratio":0.893432043696889,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":86528,"estimated_new_tokens":10321,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":215061,"pending_prefill_tokens":14874,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":7680,"new_prefill":89169,"score_linear":255023.0,"score_lmetric":416172},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":4096,"new_prefill":92753,"score_linear":68512.0,"score_lmetric":92753},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2235,"cache_hit":0,"new_prefill":96849,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":7680,"new_prefill":89169,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":96849,"score_linear":107292.0,"score_lmetric":110957},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":96849,"score_linear":361577.0,"score_lmetric":525910},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1851,"cache_hit":86528,"new_prefill":10321,"score_linear":-86528.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":167853,"ongoing_decode_tokens":0,"pending_prefill_tokens":64941,"num_requests":2,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":7680,"new_prefill":89169,"score_linear":160173.0,"score_lmetric":308220}],"chosen_score_linear":-86528.0,"chosen_score_lmetric":0,"t_first_token":282182.612331169,"t_first_token_unix":1779713047.6262746,"t_done":282187.168601589,"t_done_unix":1779713052.1825452},{"request_id":"1340290:12:1421294:835","session_id":"1340290","input_length":35303,"t_proxy_recv":282190.560598856,"t_decision_unix":1779713055.5745401,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9862051383735093,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":487,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":215061,"ongoing_decode_tokens":215061,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":35303,"score_linear":215061.0,"score_lmetric":70606},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1765,"cache_hit":0,"new_prefill":35303,"score_linear":72608.0,"score_lmetric":35303},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37398,"ongoing_decode_tokens":37398,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2235,"cache_hit":0,"new_prefill":35303,"score_linear":37398.0,"score_lmetric":35303},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32179,"ongoing_decode_tokens":32179,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":0,"new_prefill":35303,"score_linear":32179.0,"score_lmetric":35303},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":35303,"score_linear":107292.0,"score_lmetric":49411},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":268537,"ongoing_decode_tokens":75372,"pending_prefill_tokens":8333,"num_requests":4,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":0,"new_prefill":35303,"score_linear":268537.0,"score_lmetric":174544},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":34816,"new_prefill":487,"score_linear":-34816.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":167853,"ongoing_decode_tokens":124090,"pending_prefill_tokens":2803,"num_requests":2,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":35303,"score_linear":167853.0,"score_lmetric":76212}],"chosen_score_linear":-34816.0,"chosen_score_lmetric":0,"t_first_token":282190.632193565,"t_first_token_unix":1779713055.6461368,"t_done":282192.07586153,"t_done_unix":1779713057.089805},{"request_id":"1362265:9:1422050:838","session_id":"1362265","input_length":11879,"t_proxy_recv":282193.386405893,"t_decision_unix":1779713058.4003472,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9051266941661756,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":10752,"estimated_new_tokens":1127,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":215061,"ongoing_decode_tokens":215061,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":6656,"new_prefill":5223,"score_linear":208405.0,"score_lmetric":10446},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":201250,"ongoing_decode_tokens":72608,"pending_prefill_tokens":130,"num_requests":2,"active_p_offloads":0,"cached_blocks":1765,"cache_hit":10752,"new_prefill":1127,"score_linear":190498.0,"score_lmetric":2514},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37398,"ongoing_decode_tokens":37398,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2235,"cache_hit":0,"new_prefill":11879,"score_linear":37398.0,"score_lmetric":11879},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32179,"ongoing_decode_tokens":32179,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":6656,"new_prefill":5223,"score_linear":25523.0,"score_lmetric":5223},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":11879,"score_linear":107292.0,"score_lmetric":25987},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":268537,"ongoing_decode_tokens":195349,"pending_prefill_tokens":4580,"num_requests":4,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":0,"new_prefill":11879,"score_linear":268537.0,"score_lmetric":65836},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1873,"cache_hit":0,"new_prefill":11879,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":167853,"ongoing_decode_tokens":124090,"pending_prefill_tokens":2803,"num_requests":2,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":11879,"score_linear":167853.0,"score_lmetric":29364}],"chosen_score_linear":190498.0,"chosen_score_lmetric":2514,"t_first_token":282193.769403751,"t_first_token_unix":1779713058.7833474,"t_done":282194.403861336,"t_done_unix":1779713059.4178047},{"request_id":"1366128:28:1421998:837","session_id":"1366128","input_length":128642,"t_proxy_recv":282193.206762676,"t_decision_unix":1779713058.220704,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":128512,"affinity_cache_ratio":0.9989894435720837,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":128512,"estimated_new_tokens":130,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":215061,"ongoing_decode_tokens":215061,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":128642,"score_linear":215061.0,"score_lmetric":257284},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1765,"cache_hit":128512,"new_prefill":130,"score_linear":-55904.0,"score_lmetric":130},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37398,"ongoing_decode_tokens":37398,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2235,"cache_hit":0,"new_prefill":128642,"score_linear":37398.0,"score_lmetric":128642},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32179,"ongoing_decode_tokens":32179,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":0,"new_prefill":128642,"score_linear":32179.0,"score_lmetric":128642},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":128642,"score_linear":107292.0,"score_lmetric":142750},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":268537,"ongoing_decode_tokens":195349,"pending_prefill_tokens":4580,"num_requests":4,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":0,"new_prefill":128642,"score_linear":268537.0,"score_lmetric":532888},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1873,"cache_hit":0,"new_prefill":128642,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":167853,"ongoing_decode_tokens":124090,"pending_prefill_tokens":2803,"num_requests":2,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":128642,"score_linear":167853.0,"score_lmetric":262890}],"chosen_score_linear":-55904.0,"chosen_score_lmetric":130,"t_first_token":282193.555688287,"t_first_token_unix":1779713058.5696318,"t_done":282194.606234088,"t_done_unix":1779713059.6201775},{"request_id":"1270606:2:1411654:817","session_id":"1270606","input_length":107292,"t_proxy_recv":282156.808068717,"t_decision_unix":1779713021.8220098,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":93184,"affinity_cache_ratio":0.8685083696827349,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":93184,"estimated_new_tokens":14108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":107292,"score_linear":262703.0,"score_lmetric":501376},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117746,"ongoing_decode_tokens":0,"pending_prefill_tokens":8178,"num_requests":1,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":0,"new_prefill":107292,"score_linear":117746.0,"score_lmetric":115470},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2198,"cache_hit":0,"new_prefill":107292,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":107292,"score_linear":106143.0,"score_lmetric":107292},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":93184,"new_prefill":14108,"score_linear":-93184.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":107292,"score_linear":361577.0,"score_lmetric":578125},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":30154,"ongoing_decode_tokens":30154,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":107292,"score_linear":30154.0,"score_lmetric":107292},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":62138,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":1536,"new_prefill":105756,"score_linear":122554.0,"score_lmetric":167894}],"chosen_score_linear":-93184.0,"chosen_score_lmetric":0,"t_first_token":282194.399556464,"t_first_token_unix":1779713059.4134998,"t_done":282195.74263968,"t_done_unix":1779713060.756583},{"request_id":"1313181:8:1329951:394","session_id":"1313181","input_length":124090,"t_proxy_recv":282153.674402013,"t_decision_unix":1779713018.6883435,"policy":"unified","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":61952,"affinity_cache_ratio":0.499250543960029,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":61952,"estimated_new_tokens":62138,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":124090,"score_linear":262703.0,"score_lmetric":568568},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":0,"new_prefill":124090,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2194,"cache_hit":0,"new_prefill":124090,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":124090,"score_linear":106143.0,"score_lmetric":124090},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":124090,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":12800,"new_prefill":111290,"score_linear":348777.0,"score_lmetric":598115},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":114503,"ongoing_decode_tokens":114503,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":0,"new_prefill":124090,"score_linear":114503.0,"score_lmetric":372270},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":61952,"new_prefill":62138,"score_linear":-61952.0,"score_lmetric":0}],"chosen_score_linear":-61952.0,"chosen_score_lmetric":0,"t_first_token":282189.107540052,"t_first_token_unix":1779713054.1214836,"t_done":282197.146737039,"t_done_unix":1779713062.160681},{"request_id":"1301929:3:1417873:832","session_id":"1301929","input_length":37398,"t_proxy_recv":282178.507723698,"t_decision_unix":1779713043.5216649,"policy":"unified","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":2,"affinity_cache_hit":37376,"affinity_cache_ratio":0.9994117332477672,"affinity_num_requests":5,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":37398,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":235027,"pending_prefill_tokens":5660,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":3584,"new_prefill":33814,"score_linear":259119.0,"score_lmetric":157896},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":201041,"ongoing_decode_tokens":72608,"pending_prefill_tokens":433,"num_requests":2,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":37398,"score_linear":201041.0,"score_lmetric":75662},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2235,"cache_hit":0,"new_prefill":37398,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":0,"new_prefill":37398,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":37398,"score_linear":107292.0,"score_lmetric":51506},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":37376,"new_prefill":22,"score_linear":324201.0,"score_lmetric":41775},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":96849,"ongoing_decode_tokens":0,"pending_prefill_tokens":10321,"num_requests":1,"active_p_offloads":0,"cached_blocks":1851,"cache_hit":0,"new_prefill":37398,"score_linear":96849.0,"score_lmetric":47719},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":167853,"ongoing_decode_tokens":0,"pending_prefill_tokens":64941,"num_requests":2,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":37398,"score_linear":167853.0,"score_lmetric":204678}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282184.1600435,"t_first_token_unix":1779713049.1739898,"t_done":282197.469962734,"t_done_unix":1779713062.4839087},{"request_id":"1313181:8:1385690:686","session_id":"1313181","input_length":62444,"t_proxy_recv":282197.157229402,"t_decision_unix":1779713062.1711712,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":61952,"affinity_cache_ratio":0.9921209403625648,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":61952,"estimated_new_tokens":492,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":215061,"ongoing_decode_tokens":215061,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":62444,"score_linear":215061.0,"score_lmetric":124888},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":0,"new_prefill":62444,"score_linear":72608.0,"score_lmetric":62444},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37398,"ongoing_decode_tokens":37398,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2235,"cache_hit":0,"new_prefill":62444,"score_linear":37398.0,"score_lmetric":62444},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32179,"ongoing_decode_tokens":32179,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":0,"new_prefill":62444,"score_linear":32179.0,"score_lmetric":62444},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":62444,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":268537,"ongoing_decode_tokens":195349,"pending_prefill_tokens":4580,"num_requests":4,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":12800,"new_prefill":49644,"score_linear":255737.0,"score_lmetric":216896},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1873,"cache_hit":0,"new_prefill":62444,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1547,"cache_hit":61952,"new_prefill":492,"score_linear":-18189.0,"score_lmetric":492}],"chosen_score_linear":-18189.0,"chosen_score_lmetric":492,"t_first_token":282197.254146981,"t_first_token_unix":1779713062.2680907,"t_done":282197.957803374,"t_done_unix":1779713062.971747},{"request_id":"1364128:3:1419644:833","session_id":"1364128","input_length":32179,"t_proxy_recv":282184.688728592,"t_decision_unix":1779713049.7026699,"policy":"unified","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":3,"affinity_cache_hit":8704,"affinity_cache_ratio":0.2704869635476553,"affinity_num_requests":3,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":6656,"estimated_new_tokens":25523,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":242737,"ongoing_decode_tokens":242737,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1540,"cache_hit":8704,"new_prefill":23475,"score_linear":234033.0,"score_lmetric":70425},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1765,"cache_hit":6656,"new_prefill":25523,"score_linear":65952.0,"score_lmetric":25523},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37398,"ongoing_decode_tokens":37398,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2235,"cache_hit":0,"new_prefill":32179,"score_linear":37398.0,"score_lmetric":32179},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":6656,"new_prefill":25523,"score_linear":-6656.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":32179,"score_linear":107292.0,"score_lmetric":46287},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":268537,"ongoing_decode_tokens":75372,"pending_prefill_tokens":8333,"num_requests":4,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":0,"new_prefill":32179,"score_linear":268537.0,"score_lmetric":162048},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":96849,"ongoing_decode_tokens":96849,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1851,"cache_hit":0,"new_prefill":32179,"score_linear":96849.0,"score_lmetric":32179},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":167853,"ongoing_decode_tokens":0,"pending_prefill_tokens":64941,"num_requests":2,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":32179,"score_linear":167853.0,"score_lmetric":194240}],"chosen_score_linear":-6656.0,"chosen_score_lmetric":0,"t_first_token":282189.073458083,"t_first_token_unix":1779713054.0874016,"t_done":282199.79288441,"t_done_unix":1779713064.8068278},{"request_id":"1264679:2:1405259:777","session_id":"1264679","input_length":113258,"t_proxy_recv":282134.334708272,"t_decision_unix":1779712999.3486495,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":110080,"affinity_cache_ratio":0.9719401719966801,"affinity_num_requests":2,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":110080,"estimated_new_tokens":3178,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":113019,"ongoing_decode_tokens":113019,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":110080,"new_prefill":3178,"score_linear":2939.0,"score_lmetric":6356},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68109,"ongoing_decode_tokens":68109,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1655,"cache_hit":0,"new_prefill":113258,"score_linear":68109.0,"score_lmetric":113258},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":145650,"ongoing_decode_tokens":21009,"pending_prefill_tokens":5345,"num_requests":3,"active_p_offloads":0,"cached_blocks":2107,"cache_hit":0,"new_prefill":113258,"score_linear":145650.0,"score_lmetric":355809},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":117646,"ongoing_decode_tokens":106143,"pending_prefill_tokens":11503,"num_requests":2,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":113258,"score_linear":117646.0,"score_lmetric":249522},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":113258,"score_linear":21684.0,"score_lmetric":113258},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168412,"ongoing_decode_tokens":0,"pending_prefill_tokens":98268,"num_requests":3,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":113258,"score_linear":168412.0,"score_lmetric":634578},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32944,"ongoing_decode_tokens":0,"pending_prefill_tokens":32944,"num_requests":1,"active_p_offloads":0,"cached_blocks":1691,"cache_hit":0,"new_prefill":113258,"score_linear":32944.0,"score_lmetric":146202},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":113258,"score_linear":121811.0,"score_lmetric":173117}],"chosen_score_linear":2939.0,"chosen_score_lmetric":6356,"t_first_token":282176.029684346,"t_first_token_unix":1779713041.0436275,"t_done":282201.316667473,"t_done_unix":1779713066.3306115},{"request_id":"1363943:11:1424840:842","session_id":"1363943","input_length":25225,"t_proxy_recv":282203.220298242,"t_decision_unix":1779713068.2342396,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":24576,"affinity_cache_ratio":0.9742715559960357,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":24576,"estimated_new_tokens":649,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":8704,"new_prefill":16521,"score_linear":93099.0,"score_lmetric":16521},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":198806,"ongoing_decode_tokens":198806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":24064,"new_prefill":1161,"score_linear":174742.0,"score_lmetric":2322},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":25225,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":4096,"new_prefill":21129,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":129673,"num_requests":1,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":25225,"score_linear":129673.0,"score_lmetric":154898},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":268537,"ongoing_decode_tokens":195349,"pending_prefill_tokens":4580,"num_requests":4,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":0,"new_prefill":25225,"score_linear":268537.0,"score_lmetric":119220},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":101432,"ongoing_decode_tokens":101432,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1873,"cache_hit":24576,"new_prefill":649,"score_linear":76856.0,"score_lmetric":649},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1548,"cache_hit":4096,"new_prefill":21129,"score_linear":39667.0,"score_lmetric":21129}],"chosen_score_linear":76856.0,"chosen_score_lmetric":649,"t_first_token":282203.328361256,"t_first_token_unix":1779713068.3423047,"t_done":282205.335218755,"t_done_unix":1779713070.349162},{"request_id":"1399948:8:1425345:844","session_id":"1399948","input_length":11736,"t_proxy_recv":282205.1023264,"t_decision_unix":1779713070.1162677,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":11264,"affinity_cache_ratio":0.9597818677573279,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":472,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":11736,"score_linear":101803.0,"score_lmetric":11736},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":198806,"ongoing_decode_tokens":198806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":0,"new_prefill":11736,"score_linear":198806.0,"score_lmetric":23472},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":11736,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":0,"new_prefill":11736,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":129673,"num_requests":1,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":11736,"score_linear":129673.0,"score_lmetric":141409},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":268537,"ongoing_decode_tokens":195349,"pending_prefill_tokens":4580,"num_requests":4,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":0,"new_prefill":11736,"score_linear":268537.0,"score_lmetric":65264},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":126657,"ongoing_decode_tokens":126657,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1873,"cache_hit":11264,"new_prefill":472,"score_linear":115393.0,"score_lmetric":944},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":83552,"ongoing_decode_tokens":43763,"pending_prefill_tokens":1901,"num_requests":2,"active_p_offloads":0,"cached_blocks":1548,"cache_hit":0,"new_prefill":11736,"score_linear":83552.0,"score_lmetric":27274}],"chosen_score_linear":115393.0,"chosen_score_lmetric":944,"t_first_token":282205.195509103,"t_first_token_unix":1779713070.2094529,"t_done":282205.506787109,"t_done_unix":1779713070.5207303},{"request_id":"1364090:9:1423736:840","session_id":"1364090","input_length":101432,"t_proxy_recv":282199.49571757,"t_decision_unix":1779713064.5096588,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":97280,"affinity_cache_ratio":0.9590661724110734,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":97280,"estimated_new_tokens":4152,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":215061,"ongoing_decode_tokens":215061,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":7680,"new_prefill":93752,"score_linear":207381.0,"score_lmetric":187504},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":4096,"new_prefill":97336,"score_linear":68512.0,"score_lmetric":97336},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":101432,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32179,"ongoing_decode_tokens":32179,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":7680,"new_prefill":93752,"score_linear":24499.0,"score_lmetric":93752},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":129673,"num_requests":1,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":101432,"score_linear":129673.0,"score_lmetric":231105},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":268537,"ongoing_decode_tokens":195349,"pending_prefill_tokens":4580,"num_requests":4,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":0,"new_prefill":101432,"score_linear":268537.0,"score_lmetric":424048},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1873,"cache_hit":97280,"new_prefill":4152,"score_linear":-97280.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1548,"cache_hit":7680,"new_prefill":93752,"score_linear":36083.0,"score_lmetric":93752}],"chosen_score_linear":-97280.0,"chosen_score_lmetric":0,"t_first_token":282202.191074278,"t_first_token_unix":1779713067.205018,"t_done":282206.234507571,"t_done_unix":1779713071.248451},{"request_id":"1355951:5:1425005:843","session_id":"1355951","input_length":39789,"t_proxy_recv":282203.875135568,"t_decision_unix":1779713068.8890767,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":37888,"affinity_cache_ratio":0.9522229761994521,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":37888,"estimated_new_tokens":1901,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":39789,"score_linear":101803.0,"score_lmetric":39789},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":198806,"ongoing_decode_tokens":198806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":0,"new_prefill":39789,"score_linear":198806.0,"score_lmetric":79578},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":34816,"new_prefill":4973,"score_linear":-34816.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":0,"new_prefill":39789,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":129673,"num_requests":1,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":39789,"score_linear":129673.0,"score_lmetric":169462},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":268537,"ongoing_decode_tokens":195349,"pending_prefill_tokens":4580,"num_requests":4,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":0,"new_prefill":39789,"score_linear":268537.0,"score_lmetric":177476},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":126657,"ongoing_decode_tokens":126657,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1873,"cache_hit":0,"new_prefill":39789,"score_linear":126657.0,"score_lmetric":79578},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1548,"cache_hit":37888,"new_prefill":1901,"score_linear":5875.0,"score_lmetric":1901}],"chosen_score_linear":5875.0,"chosen_score_lmetric":1901,"t_first_token":282210.154655685,"t_first_token_unix":1779713075.1685996,"t_done":282210.356970166,"t_done_unix":1779713075.3709133},{"request_id":"1355484:14:1398178:740","session_id":"1355484","input_length":31224,"t_proxy_recv":282109.752580211,"t_decision_unix":1779712974.7665217,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":28672,"affinity_cache_ratio":0.9182679989751473,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":28672,"estimated_new_tokens":2552,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":31224,"score_linear":101803.0,"score_lmetric":59811},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":13134,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1597,"cache_hit":0,"new_prefill":31224,"score_linear":13134.0,"score_lmetric":31224},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16550,"ongoing_decode_tokens":0,"pending_prefill_tokens":16550,"num_requests":1,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":0,"new_prefill":31224,"score_linear":16550.0,"score_lmetric":47774},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1776,"cache_hit":0,"new_prefill":31224,"score_linear":106143.0,"score_lmetric":31224},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":33321,"ongoing_decode_tokens":33321,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":31224,"score_linear":33321.0,"score_lmetric":31224},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":93040,"num_requests":1,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":28672,"new_prefill":2552,"score_linear":64368.0,"score_lmetric":95592},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":3213,"ongoing_decode_tokens":0,"pending_prefill_tokens":3213,"num_requests":1,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":31224,"score_linear":3213.0,"score_lmetric":34437},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":162939,"ongoing_decode_tokens":162939,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":26624,"new_prefill":4600,"score_linear":136315.0,"score_lmetric":9200}],"chosen_score_linear":64368.0,"chosen_score_lmetric":95592,"t_first_token":282136.498201373,"t_first_token_unix":1779713001.512144,"t_done":282210.993547621,"t_done_unix":1779713076.007491},{"request_id":"1268861:8:1403753:765","session_id":"1268861","input_length":44148,"t_proxy_recv":282129.138847274,"t_decision_unix":1779712994.1527884,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":41472,"affinity_cache_ratio":0.939385702636586,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":41472,"estimated_new_tokens":2676,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":0,"new_prefill":44148,"score_linear":101803.0,"score_lmetric":44148},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":169473,"ongoing_decode_tokens":101364,"pending_prefill_tokens":17421,"num_requests":3,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":0,"new_prefill":44148,"score_linear":169473.0,"score_lmetric":184707},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2104,"cache_hit":0,"new_prefill":44148,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":0,"new_prefill":44148,"score_linear":106143.0,"score_lmetric":44148},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":21504,"new_prefill":22644,"score_linear":-21504.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":124264,"ongoing_decode_tokens":0,"pending_prefill_tokens":95592,"num_requests":2,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":41472,"new_prefill":2676,"score_linear":82792.0,"score_lmetric":196536},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":87072,"ongoing_decode_tokens":87072,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":44148,"score_linear":87072.0,"score_lmetric":88296},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":44148,"score_linear":121811.0,"score_lmetric":104007}],"chosen_score_linear":82792.0,"chosen_score_lmetric":196536,"t_first_token":282144.302652987,"t_first_token_unix":1779713009.3165965,"t_done":282212.152710626,"t_done_unix":1779713077.166654},{"request_id":"1366128:29:1424316:841","session_id":"1366128","input_length":126198,"t_proxy_recv":282201.400586903,"t_decision_unix":1779713066.414528,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":125952,"affinity_cache_ratio":0.9980506822612085,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":125952,"estimated_new_tokens":246,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":126198,"score_linear":101803.0,"score_lmetric":126198},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":125952,"new_prefill":246,"score_linear":-53344.0,"score_lmetric":246},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":126198,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":0,"new_prefill":126198,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":129673,"num_requests":1,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":126198,"score_linear":129673.0,"score_lmetric":255871},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":268537,"ongoing_decode_tokens":195349,"pending_prefill_tokens":4580,"num_requests":4,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":0,"new_prefill":126198,"score_linear":268537.0,"score_lmetric":523112},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":101432,"ongoing_decode_tokens":0,"pending_prefill_tokens":4152,"num_requests":1,"active_p_offloads":0,"cached_blocks":1873,"cache_hit":0,"new_prefill":126198,"score_linear":101432.0,"score_lmetric":130350},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1548,"cache_hit":0,"new_prefill":126198,"score_linear":43763.0,"score_lmetric":126198}],"chosen_score_linear":-53344.0,"chosen_score_lmetric":246,"t_first_token":282201.569895829,"t_first_token_unix":1779713066.5838397,"t_done":282212.167709328,"t_done_unix":1779713077.1816528},{"request_id":"1363093:5:1407810:790","session_id":"1363093","input_length":73188,"t_proxy_recv":282143.313816888,"t_decision_unix":1779713008.3277578,"policy":"unified","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":68608,"affinity_cache_ratio":0.9374214352079576,"affinity_num_requests":4,"avg_num_requests":1.75,"fallback_score":44036,"tie_break_used":false,"cache_hit":68608,"estimated_new_tokens":4580,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":235027,"ongoing_decode_tokens":101803,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":73188,"score_linear":235027.0,"score_lmetric":256740},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":179082,"ongoing_decode_tokens":69508,"pending_prefill_tokens":11782,"num_requests":2,"active_p_offloads":0,"cached_blocks":1689,"cache_hit":0,"new_prefill":73188,"score_linear":179082.0,"score_lmetric":169940},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":4789,"ongoing_decode_tokens":4789,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":73188,"score_linear":4789.0,"score_lmetric":73188},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":73188,"score_linear":106143.0,"score_lmetric":73188},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":1501,"ongoing_decode_tokens":1501,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1860,"cache_hit":0,"new_prefill":73188,"score_linear":1501.0,"score_lmetric":73188},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":288389,"ongoing_decode_tokens":124264,"pending_prefill_tokens":6429,"num_requests":4,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":68608,"new_prefill":4580,"score_linear":219781.0,"score_lmetric":44036},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32944,"ongoing_decode_tokens":32944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1702,"cache_hit":0,"new_prefill":73188,"score_linear":32944.0,"score_lmetric":73188},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":73188,"score_linear":121811.0,"score_lmetric":133047}],"chosen_score_linear":219781.0,"chosen_score_lmetric":44036,"t_first_token":282210.929265401,"t_first_token_unix":1779713075.943209,"t_done":282212.18830179,"t_done_unix":1779713077.2022448},{"request_id":"1381387:7:1411928:819","session_id":"1381387","input_length":72608,"t_proxy_recv":282157.631792652,"t_decision_unix":1779713022.6457338,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":71680,"affinity_cache_ratio":0.9872190392243279,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":71680,"estimated_new_tokens":928,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":72608,"score_linear":262703.0,"score_lmetric":362640},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117746,"ongoing_decode_tokens":0,"pending_prefill_tokens":8178,"num_requests":1,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":71680,"new_prefill":928,"score_linear":46066.0,"score_lmetric":9106},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2198,"cache_hit":0,"new_prefill":72608,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":72608,"score_linear":106143.0,"score_lmetric":72608},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":0,"new_prefill":72608,"score_linear":107292.0,"score_lmetric":86716},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":72608,"score_linear":361577.0,"score_lmetric":404705},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":30154,"ongoing_decode_tokens":30154,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":512,"new_prefill":72096,"score_linear":29642.0,"score_lmetric":72096},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":62138,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":0,"new_prefill":72608,"score_linear":124090.0,"score_lmetric":134746}],"chosen_score_linear":46066.0,"chosen_score_lmetric":9106,"t_first_token":282160.886349549,"t_first_token_unix":1779713025.9002929,"t_done":282212.435322086,"t_done_unix":1779713077.4492652},{"request_id":"1270340:5:1417075:829","session_id":"1270340","input_length":43763,"t_proxy_recv":282175.877073045,"t_decision_unix":1779713040.8910143,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":40960,"affinity_cache_ratio":0.9359504604346137,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":40960,"estimated_new_tokens":2803,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":262703,"ongoing_decode_tokens":101803,"pending_prefill_tokens":18052,"num_requests":4,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":43763,"score_linear":262703.0,"score_lmetric":247260},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":43763,"score_linear":72608.0,"score_lmetric":43763},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2235,"cache_hit":0,"new_prefill":43763,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":0,"new_prefill":43763,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1900,"cache_hit":1536,"new_prefill":42227,"score_linear":105756.0,"score_lmetric":56335},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":361577,"ongoing_decode_tokens":168412,"pending_prefill_tokens":8333,"num_requests":5,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":43763,"score_linear":361577.0,"score_lmetric":260480},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":126523,"ongoing_decode_tokens":126523,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":43763,"score_linear":126523.0,"score_lmetric":87526},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":62138,"num_requests":1,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":40960,"new_prefill":2803,"score_linear":83130.0,"score_lmetric":64941}],"chosen_score_linear":83130.0,"chosen_score_lmetric":64941,"t_first_token":282196.184147913,"t_first_token_unix":1779713061.1980913,"t_done":282214.179486373,"t_done_unix":1779713079.1934297},{"request_id":"1275274:2:1405679:780","session_id":"1275274","input_length":119977,"t_proxy_recv":282135.806291861,"t_decision_unix":1779713000.820233,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":116224,"affinity_cache_ratio":0.9687190044758579,"affinity_num_requests":3,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":116224,"estimated_new_tokens":3753,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":235027,"ongoing_decode_tokens":101803,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1518,"cache_hit":0,"new_prefill":119977,"score_linear":235027.0,"score_lmetric":397107},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":69508,"ongoing_decode_tokens":69508,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1689,"cache_hit":0,"new_prefill":119977,"score_linear":69508.0,"score_lmetric":119977},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":145650,"ongoing_decode_tokens":21009,"pending_prefill_tokens":5345,"num_requests":3,"active_p_offloads":0,"cached_blocks":2107,"cache_hit":0,"new_prefill":119977,"score_linear":145650.0,"score_lmetric":375966},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":119977,"score_linear":106143.0,"score_lmetric":119977},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":119977,"score_linear":21684.0,"score_lmetric":119977},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168412,"ongoing_decode_tokens":0,"pending_prefill_tokens":98268,"num_requests":3,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":116224,"new_prefill":3753,"score_linear":52188.0,"score_lmetric":306063},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":39093,"ongoing_decode_tokens":0,"pending_prefill_tokens":34485,"num_requests":2,"active_p_offloads":0,"cached_blocks":1691,"cache_hit":0,"new_prefill":119977,"score_linear":39093.0,"score_lmetric":308924},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":119977,"score_linear":121811.0,"score_lmetric":179836}],"chosen_score_linear":52188.0,"chosen_score_lmetric":306063,"t_first_token":282192.660566352,"t_first_token_unix":1779713057.67451,"t_done":282214.240450872,"t_done_unix":1779713079.254394},{"request_id":"1277428:8:1426669:846","session_id":"1277428","input_length":42342,"t_proxy_recv":282209.77693205,"t_decision_unix":1779713074.7908733,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9552690000472345,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":1894,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":42342,"score_linear":101803.0,"score_lmetric":42342},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":198806,"ongoing_decode_tokens":198806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":4608,"new_prefill":37734,"score_linear":194198.0,"score_lmetric":75468},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":42342,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":40448,"new_prefill":1894,"score_linear":-40448.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":129673,"num_requests":1,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":42342,"score_linear":129673.0,"score_lmetric":172015},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":268537,"ongoing_decode_tokens":195349,"pending_prefill_tokens":4580,"num_requests":4,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":0,"new_prefill":42342,"score_linear":268537.0,"score_lmetric":187688},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":26923,"ongoing_decode_tokens":0,"pending_prefill_tokens":1835,"num_requests":1,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":0,"new_prefill":42342,"score_linear":26923.0,"score_lmetric":44177},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":83552,"ongoing_decode_tokens":43763,"pending_prefill_tokens":1901,"num_requests":2,"active_p_offloads":0,"cached_blocks":1548,"cache_hit":0,"new_prefill":42342,"score_linear":83552.0,"score_lmetric":88486}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":282210.42739801,"t_first_token_unix":1779713075.4413416,"t_done":282214.73655956,"t_done_unix":1779713079.7505026},{"request_id":"1362265:10:1427606:850","session_id":"1362265","input_length":12596,"t_proxy_recv":282212.796899871,"t_decision_unix":1779713077.8108413,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":11776,"affinity_cache_ratio":0.934899968243887,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":820,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":6656,"new_prefill":5940,"score_linear":95147.0,"score_lmetric":5940},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":11776,"new_prefill":820,"score_linear":-11776.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":31469,"ongoing_decode_tokens":0,"pending_prefill_tokens":31469,"num_requests":1,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":12596,"score_linear":31469.0,"score_lmetric":44065},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":42342,"ongoing_decode_tokens":42342,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":6656,"new_prefill":5940,"score_linear":35686.0,"score_lmetric":5940},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":237991,"ongoing_decode_tokens":0,"pending_prefill_tokens":130983,"num_requests":2,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":12596,"score_linear":237991.0,"score_lmetric":287158},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":169285,"ongoing_decode_tokens":119977,"pending_prefill_tokens":5276,"num_requests":2,"active_p_offloads":0,"cached_blocks":2154,"cache_hit":0,"new_prefill":12596,"score_linear":169285.0,"score_lmetric":35744},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":134943,"ongoing_decode_tokens":26923,"pending_prefill_tokens":6644,"num_requests":2,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":0,"new_prefill":12596,"score_linear":134943.0,"score_lmetric":38480},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":12596,"score_linear":43763.0,"score_lmetric":12596}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":0,"t_first_token":282212.936875103,"t_first_token_unix":1779713077.9508185,"t_done":282214.854854056,"t_done_unix":1779713079.8687973},{"request_id":"1268861:9:1421319:836","session_id":"1268861","input_length":49308,"t_proxy_recv":282212.174083065,"t_decision_unix":1779713077.1880245,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":44032,"affinity_cache_ratio":0.8929991076498742,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":44032,"estimated_new_tokens":5276,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":49308,"score_linear":101803.0,"score_lmetric":49308},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1768,"cache_hit":0,"new_prefill":49308,"score_linear":72608.0,"score_lmetric":49308},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":31469,"ongoing_decode_tokens":0,"pending_prefill_tokens":31469,"num_requests":1,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":49308,"score_linear":31469.0,"score_lmetric":80777},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":42342,"ongoing_decode_tokens":42342,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":0,"new_prefill":49308,"score_linear":42342.0,"score_lmetric":49308},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":237991,"ongoing_decode_tokens":0,"pending_prefill_tokens":130983,"num_requests":2,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":21504,"new_prefill":27804,"score_linear":216487.0,"score_lmetric":317574},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":193165,"ongoing_decode_tokens":193165,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2145,"cache_hit":44032,"new_prefill":5276,"score_linear":149133.0,"score_lmetric":10552},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":134943,"ongoing_decode_tokens":26923,"pending_prefill_tokens":6644,"num_requests":2,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":0,"new_prefill":49308,"score_linear":134943.0,"score_lmetric":111904},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":49308,"score_linear":43763.0,"score_lmetric":49308}],"chosen_score_linear":149133.0,"chosen_score_lmetric":10552,"t_first_token":282213.750728367,"t_first_token_unix":1779713078.7646728,"t_done":282215.680535646,"t_done_unix":1779713080.694479},{"request_id":"1355484:15:1400281:750","session_id":"1355484","input_length":31469,"t_proxy_recv":282211.000421,"t_decision_unix":1779713076.0143623,"policy":"unified","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":2,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9924687787981823,"affinity_num_requests":3,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":31469,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":31469,"score_linear":101803.0,"score_lmetric":31469},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":198806,"ongoing_decode_tokens":198806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":0,"new_prefill":31469,"score_linear":198806.0,"score_lmetric":62938},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":31469,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":42342,"ongoing_decode_tokens":42342,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":0,"new_prefill":31469,"score_linear":42342.0,"score_lmetric":31469},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":129673,"num_requests":1,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":31469,"score_linear":129673.0,"score_lmetric":161142},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":237313,"ongoing_decode_tokens":237313,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":2140,"cache_hit":31232,"new_prefill":237,"score_linear":206081.0,"score_lmetric":711},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":134943,"ongoing_decode_tokens":26923,"pending_prefill_tokens":6644,"num_requests":2,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":0,"new_prefill":31469,"score_linear":134943.0,"score_lmetric":76226},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":26624,"new_prefill":4845,"score_linear":17139.0,"score_lmetric":4845}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282215.244150185,"t_first_token_unix":1779713080.258094,"t_done":282215.746990692,"t_done_unix":1779713080.760934},{"request_id":"1355484:16:1401320:758","session_id":"1355484","input_length":32332,"t_proxy_recv":282215.754262893,"t_decision_unix":1779713080.7682045,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9659779784733391,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":31232,"estimated_new_tokens":1100,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":32332,"score_linear":101803.0,"score_lmetric":32332},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":32332,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2365,"cache_hit":31232,"new_prefill":1100,"score_linear":-31232.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":32332,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":237991,"ongoing_decode_tokens":0,"pending_prefill_tokens":130983,"num_requests":2,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":32332,"score_linear":237991.0,"score_lmetric":326630},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":31232,"new_prefill":1100,"score_linear":-31232.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":134943,"ongoing_decode_tokens":134943,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":0,"new_prefill":32332,"score_linear":134943.0,"score_lmetric":64664},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":26624,"new_prefill":5708,"score_linear":-26624.0,"score_lmetric":0}],"chosen_score_linear":-31232.0,"chosen_score_lmetric":0,"t_first_token":282216.016088694,"t_first_token_unix":1779713081.0300322,"t_done":282219.890986397,"t_done_unix":1779713084.9049299},{"request_id":"1363943:12:1426646:845","session_id":"1363943","input_length":26923,"t_proxy_recv":282209.693721319,"t_decision_unix":1779713074.7076623,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":25088,"affinity_cache_ratio":0.9318426624076068,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":25088,"estimated_new_tokens":1835,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":8704,"new_prefill":18219,"score_linear":93099.0,"score_lmetric":18219},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":198806,"ongoing_decode_tokens":198806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":24064,"new_prefill":2859,"score_linear":174742.0,"score_lmetric":5718},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":26923,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":4096,"new_prefill":22827,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":129673,"num_requests":1,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":26923,"score_linear":129673.0,"score_lmetric":156596},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":268537,"ongoing_decode_tokens":195349,"pending_prefill_tokens":4580,"num_requests":4,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":0,"new_prefill":26923,"score_linear":268537.0,"score_lmetric":126012},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":25088,"new_prefill":1835,"score_linear":-25088.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":83552,"ongoing_decode_tokens":43763,"pending_prefill_tokens":1901,"num_requests":2,"active_p_offloads":0,"cached_blocks":1548,"cache_hit":4096,"new_prefill":22827,"score_linear":79456.0,"score_lmetric":49456}],"chosen_score_linear":-25088.0,"chosen_score_lmetric":0,"t_first_token":282210.048205503,"t_first_token_unix":1779713075.062149,"t_done":282221.409449215,"t_done_unix":1779713086.4233928},{"request_id":"1314357:5:1393659:719","session_id":"1314357","input_length":101803,"t_proxy_recv":282093.795623552,"t_decision_unix":1779712958.8095648,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":73216,"affinity_cache_ratio":0.7191929510918146,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73216,"estimated_new_tokens":28587,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1497,"cache_hit":73216,"new_prefill":28587,"score_linear":-73216.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1595,"cache_hit":0,"new_prefill":101803,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2012,"cache_hit":0,"new_prefill":101803,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":101803,"score_linear":15283.0,"score_lmetric":101803},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":101803,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":0,"new_prefill":101803,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":48944,"ongoing_decode_tokens":48944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1575,"cache_hit":0,"new_prefill":101803,"score_linear":48944.0,"score_lmetric":101803},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":245313,"ongoing_decode_tokens":100766,"pending_prefill_tokens":45219,"num_requests":3,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":101803,"score_linear":245313.0,"score_lmetric":441066}],"chosen_score_linear":-73216.0,"chosen_score_lmetric":0,"t_first_token":282127.713905908,"t_first_token_unix":1779712992.7278497,"t_done":282225.244955792,"t_done_unix":1779713090.2588995},{"request_id":"1399948:9:1433544:857","session_id":"1399948","input_length":11993,"t_proxy_recv":282233.991027207,"t_decision_unix":1779713099.0049684,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":11264,"affinity_cache_ratio":0.9392145418160593,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":729,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":231656,"ongoing_decode_tokens":105709,"pending_prefill_tokens":11771,"num_requests":2,"active_p_offloads":0,"cached_blocks":1614,"cache_hit":0,"new_prefill":11993,"score_linear":231656.0,"score_lmetric":47528},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":11993,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":40567,"ongoing_decode_tokens":40567,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2368,"cache_hit":0,"new_prefill":11993,"score_linear":40567.0,"score_lmetric":11993},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":73878,"ongoing_decode_tokens":0,"pending_prefill_tokens":73878,"num_requests":1,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":11993,"score_linear":73878.0,"score_lmetric":85871},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":239831,"ongoing_decode_tokens":0,"pending_prefill_tokens":131287,"num_requests":3,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":11993,"score_linear":239831.0,"score_lmetric":429840},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":0,"new_prefill":11993,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1886,"cache_hit":11264,"new_prefill":729,"score_linear":96756.0,"score_lmetric":729},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":11993,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":96756.0,"chosen_score_lmetric":729,"t_first_token":282234.087216835,"t_first_token_unix":1779713099.1011605,"t_done":282234.245296486,"t_done_unix":1779713099.2592397},{"request_id":"1382089:3:1433211:856","session_id":"1382089","input_length":40567,"t_proxy_recv":282232.780453865,"t_decision_unix":1779713097.7943954,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":38912,"affinity_cache_ratio":0.9592032933172283,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":38912,"estimated_new_tokens":1655,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":231656,"ongoing_decode_tokens":105709,"pending_prefill_tokens":11771,"num_requests":2,"active_p_offloads":0,"cached_blocks":1614,"cache_hit":0,"new_prefill":40567,"score_linear":231656.0,"score_lmetric":104676},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":40567,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2368,"cache_hit":38912,"new_prefill":1655,"score_linear":-38912.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":73878,"ongoing_decode_tokens":0,"pending_prefill_tokens":73878,"num_requests":1,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":40567,"score_linear":73878.0,"score_lmetric":114445},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":239831,"ongoing_decode_tokens":0,"pending_prefill_tokens":131287,"num_requests":3,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":40567,"score_linear":239831.0,"score_lmetric":515562},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":0,"new_prefill":40567,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1886,"cache_hit":0,"new_prefill":40567,"score_linear":108020.0,"score_lmetric":40567},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":34816,"new_prefill":5751,"score_linear":-34816.0,"score_lmetric":0}],"chosen_score_linear":-38912.0,"chosen_score_lmetric":0,"t_first_token":282233.256324534,"t_first_token_unix":1779713098.270268,"t_done":282235.395441245,"t_done_unix":1779713100.4093845},{"request_id":"1399948:10:1434418:858","session_id":"1399948","input_length":12032,"t_proxy_recv":282237.002366144,"t_decision_unix":1779713102.016307,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":11776,"affinity_cache_ratio":0.9787234042553191,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":256,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":231656,"ongoing_decode_tokens":231656,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1614,"cache_hit":0,"new_prefill":12032,"score_linear":231656.0,"score_lmetric":24064},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":12032,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":12032,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":73878,"ongoing_decode_tokens":0,"pending_prefill_tokens":73878,"num_requests":1,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":12032,"score_linear":73878.0,"score_lmetric":85910},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":239831,"ongoing_decode_tokens":0,"pending_prefill_tokens":131287,"num_requests":3,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":12032,"score_linear":239831.0,"score_lmetric":429957},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":0,"new_prefill":12032,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":11776,"new_prefill":256,"score_linear":96244.0,"score_lmetric":256},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":12032,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":96244.0,"chosen_score_lmetric":256,"t_first_token":282237.066973866,"t_first_token_unix":1779713102.080917,"t_done":282237.559139516,"t_done_unix":1779713102.5730827},{"request_id":"1399948:11:1435163:860","session_id":"1399948","input_length":12106,"t_proxy_recv":282239.7055596,"t_decision_unix":1779713104.719501,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":11776,"affinity_cache_ratio":0.9727407896910623,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":330,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":231656,"ongoing_decode_tokens":231656,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1614,"cache_hit":0,"new_prefill":12106,"score_linear":231656.0,"score_lmetric":24212},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":12106,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":12106,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":73878,"ongoing_decode_tokens":0,"pending_prefill_tokens":73878,"num_requests":1,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":12106,"score_linear":73878.0,"score_lmetric":85984},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":239831,"ongoing_decode_tokens":0,"pending_prefill_tokens":131287,"num_requests":3,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":12106,"score_linear":239831.0,"score_lmetric":430179},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":0,"new_prefill":12106,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":11776,"new_prefill":330,"score_linear":96244.0,"score_lmetric":330},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45269,"ongoing_decode_tokens":45269,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":12106,"score_linear":45269.0,"score_lmetric":12106}],"chosen_score_linear":96244.0,"chosen_score_lmetric":330,"t_first_token":282239.768945343,"t_first_token_unix":1779713104.7828887,"t_done":282239.889198614,"t_done_unix":1779713104.9031422},{"request_id":"1270340:6:1434904:859","session_id":"1270340","input_length":45269,"t_proxy_recv":282238.645812658,"t_decision_unix":1779713103.659754,"policy":"unified","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":45056,"affinity_cache_ratio":0.9952947933464402,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":45056,"estimated_new_tokens":213,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":231656,"ongoing_decode_tokens":231656,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1614,"cache_hit":0,"new_prefill":45269,"score_linear":231656.0,"score_lmetric":90538},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":45269,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":45269,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":73878,"ongoing_decode_tokens":0,"pending_prefill_tokens":73878,"num_requests":1,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":45269,"score_linear":73878.0,"score_lmetric":119147},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":239831,"ongoing_decode_tokens":0,"pending_prefill_tokens":131287,"num_requests":3,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":1536,"new_prefill":43733,"score_linear":238295.0,"score_lmetric":525060},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":0,"new_prefill":45269,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":45269,"score_linear":108020.0,"score_lmetric":45269},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":45056,"new_prefill":213,"score_linear":-45056.0,"score_lmetric":0}],"chosen_score_linear":-45056.0,"chosen_score_lmetric":0,"t_first_token":282238.750779956,"t_first_token_unix":1779713103.7647233,"t_done":282240.23172414,"t_done_unix":1779713105.2456672},{"request_id":"1314357:6:1422517:839","session_id":"1314357","input_length":105709,"t_proxy_recv":282225.263347383,"t_decision_unix":1779713090.277289,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":105472,"affinity_cache_ratio":0.9977579960079085,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":105472,"estimated_new_tokens":237,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1614,"cache_hit":105472,"new_prefill":237,"score_linear":-105472.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":105709,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2368,"cache_hit":0,"new_prefill":105709,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":105709,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":239831,"ongoing_decode_tokens":0,"pending_prefill_tokens":131287,"num_requests":3,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":105709,"score_linear":239831.0,"score_lmetric":710988},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":0,"new_prefill":105709,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1886,"cache_hit":0,"new_prefill":105709,"score_linear":108020.0,"score_lmetric":105709},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":105709,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105472.0,"chosen_score_lmetric":0,"t_first_token":282225.424761359,"t_first_token_unix":1779713090.4387047,"t_done":282242.622292249,"t_done_unix":1779713107.6362357},{"request_id":"1399948:12:1436179:863","session_id":"1399948","input_length":12164,"t_proxy_recv":282243.13183592,"t_decision_unix":1779713108.1457772,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":11776,"affinity_cache_ratio":0.9681025978296613,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":388,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":232226,"ongoing_decode_tokens":232226,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1615,"cache_hit":0,"new_prefill":12164,"score_linear":232226.0,"score_lmetric":24328},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":12164,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":29412,"ongoing_decode_tokens":0,"pending_prefill_tokens":29412,"num_requests":1,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":12164,"score_linear":29412.0,"score_lmetric":41576},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":73878,"ongoing_decode_tokens":0,"pending_prefill_tokens":73878,"num_requests":1,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":12164,"score_linear":73878.0,"score_lmetric":86042},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":239831,"ongoing_decode_tokens":0,"pending_prefill_tokens":131287,"num_requests":3,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":12164,"score_linear":239831.0,"score_lmetric":430353},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":0,"new_prefill":12164,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":11776,"new_prefill":388,"score_linear":96244.0,"score_lmetric":388},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":12164,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":96244.0,"chosen_score_lmetric":388,"t_first_token":282243.195739132,"t_first_token_unix":1779713108.2096825,"t_done":282243.508163638,"t_done_unix":1779713108.522107},{"request_id":"1314357:7:1427957:851","session_id":"1314357","input_length":106279,"t_proxy_recv":282242.638620381,"t_decision_unix":1779713107.6525617,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":105984,"affinity_cache_ratio":0.9972242870181315,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":105984,"estimated_new_tokens":295,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1615,"cache_hit":105984,"new_prefill":295,"score_linear":19963.0,"score_lmetric":295},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":106279,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":29412,"ongoing_decode_tokens":0,"pending_prefill_tokens":29412,"num_requests":1,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":106279,"score_linear":29412.0,"score_lmetric":135691},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":73878,"ongoing_decode_tokens":0,"pending_prefill_tokens":73878,"num_requests":1,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":106279,"score_linear":73878.0,"score_lmetric":180157},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":239831,"ongoing_decode_tokens":0,"pending_prefill_tokens":131287,"num_requests":3,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":106279,"score_linear":239831.0,"score_lmetric":712698},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":0,"new_prefill":106279,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":106279,"score_linear":108020.0,"score_lmetric":106279},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":106279,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":19963.0,"chosen_score_lmetric":295,"t_first_token":282242.862866037,"t_first_token_unix":1779713107.8768094,"t_done":282251.79548093,"t_done_unix":1779713116.8094244},{"request_id":"1342921:5:1433192:855","session_id":"1342921","input_length":73878,"t_proxy_recv":282232.701198625,"t_decision_unix":1779713097.7151399,"policy":"unified","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":3,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9633179024878854,"affinity_num_requests":3,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":73878,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":231656,"ongoing_decode_tokens":105709,"pending_prefill_tokens":11771,"num_requests":2,"active_p_offloads":0,"cached_blocks":1614,"cache_hit":0,"new_prefill":73878,"score_linear":231656.0,"score_lmetric":171298},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":73878,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2368,"cache_hit":0,"new_prefill":73878,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":73878,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":239831,"ongoing_decode_tokens":0,"pending_prefill_tokens":131287,"num_requests":3,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":71168,"new_prefill":2710,"score_linear":168663.0,"score_lmetric":401991},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":0,"new_prefill":73878,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1886,"cache_hit":0,"new_prefill":73878,"score_linear":108020.0,"score_lmetric":73878},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":73878,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282251.482252536,"t_first_token_unix":1779713116.4961958,"t_done":282252.609376642,"t_done_unix":1779713117.6233203},{"request_id":"1270606:3:1427164:848","session_id":"1270606","input_length":108318,"t_proxy_recv":282211.563467132,"t_decision_unix":1779713076.5774086,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":107008,"affinity_cache_ratio":0.987905980538784,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":107008,"estimated_new_tokens":1310,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":108318,"score_linear":101803.0,"score_lmetric":108318},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":198806,"ongoing_decode_tokens":198806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":0,"new_prefill":108318,"score_linear":198806.0,"score_lmetric":216636},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":31469,"ongoing_decode_tokens":0,"pending_prefill_tokens":31469,"num_requests":1,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":108318,"score_linear":31469.0,"score_lmetric":139787},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":42342,"ongoing_decode_tokens":42342,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":0,"new_prefill":108318,"score_linear":42342.0,"score_lmetric":108318},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":129673,"num_requests":1,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":107008,"new_prefill":1310,"score_linear":22665.0,"score_lmetric":130983},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":237313,"ongoing_decode_tokens":237313,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":2140,"cache_hit":0,"new_prefill":108318,"score_linear":237313.0,"score_lmetric":324954},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":134943,"ongoing_decode_tokens":26923,"pending_prefill_tokens":6644,"num_requests":2,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":0,"new_prefill":108318,"score_linear":134943.0,"score_lmetric":229924},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":1536,"new_prefill":106782,"score_linear":42227.0,"score_lmetric":106782}],"chosen_score_linear":22665.0,"chosen_score_lmetric":130983,"t_first_token":282252.248565932,"t_first_token_unix":1779713117.2625093,"t_done":282252.716465834,"t_done_unix":1779713117.7304091},{"request_id":"1314357:8:1431749:854","session_id":"1314357","input_length":106805,"t_proxy_recv":282251.813246845,"t_decision_unix":1779713116.8271885,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":106496,"affinity_cache_ratio":0.9971068770188661,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":106496,"estimated_new_tokens":309,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":106496,"new_prefill":309,"score_linear":19451.0,"score_lmetric":309},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":106805,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":156900,"ongoing_decode_tokens":29412,"pending_prefill_tokens":5120,"num_requests":2,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":106805,"score_linear":156900.0,"score_lmetric":223850},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":73878,"ongoing_decode_tokens":73878,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":106805,"score_linear":73878.0,"score_lmetric":106805},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":239831,"ongoing_decode_tokens":0,"pending_prefill_tokens":131287,"num_requests":3,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":106805,"score_linear":239831.0,"score_lmetric":714276},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":0,"new_prefill":106805,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":136731,"ongoing_decode_tokens":136731,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":106805,"score_linear":136731.0,"score_lmetric":213610},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":106805,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":19451.0,"chosen_score_lmetric":309,"t_first_token":282252.038895293,"t_first_token_unix":1779713117.0528386,"t_done":282254.242636068,"t_done_unix":1779713119.2565799},{"request_id":"1313181:9:1333349:415","session_id":"1313181","input_length":129673,"t_proxy_recv":282197.98164193,"t_decision_unix":1779713062.9955833,"policy":"unified","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":4,"affinity_cache_hit":62464,"affinity_cache_ratio":0.481703978468918,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":129673,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":215061,"ongoing_decode_tokens":215061,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":129673,"score_linear":215061.0,"score_lmetric":259346},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":0,"new_prefill":129673,"score_linear":72608.0,"score_lmetric":129673},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":129673,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32179,"ongoing_decode_tokens":32179,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":0,"new_prefill":129673,"score_linear":32179.0,"score_lmetric":129673},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":129673,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":268537,"ongoing_decode_tokens":195349,"pending_prefill_tokens":4580,"num_requests":4,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":12800,"new_prefill":116873,"score_linear":255737.0,"score_lmetric":485812},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1873,"cache_hit":0,"new_prefill":129673,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1548,"cache_hit":62464,"new_prefill":67209,"score_linear":-18701.0,"score_lmetric":67209}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282252.23410004,"t_first_token_unix":1779713117.2480435,"t_done":282254.282482756,"t_done_unix":1779713119.296426},{"request_id":"1268861:10:1439325:866","session_id":"1268861","input_length":52920,"t_proxy_recv":282253.659994983,"t_decision_unix":1779713118.6739361,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":49152,"affinity_cache_ratio":0.9287981859410431,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49152,"estimated_new_tokens":3768,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":232752,"ongoing_decode_tokens":232752,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":52920,"score_linear":232752.0,"score_lmetric":105840},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":52920,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":156900,"ongoing_decode_tokens":29412,"pending_prefill_tokens":5120,"num_requests":2,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":52920,"score_linear":156900.0,"score_lmetric":116080},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2104,"cache_hit":0,"new_prefill":52920,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131513,"ongoing_decode_tokens":131513,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":21504,"new_prefill":31416,"score_linear":110009.0,"score_lmetric":62832},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":49152,"new_prefill":3768,"score_linear":-49152.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":136731,"ongoing_decode_tokens":136731,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":52920,"score_linear":136731.0,"score_lmetric":105840},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":52920,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49152.0,"chosen_score_lmetric":0,"t_first_token":282254.866646452,"t_first_token_unix":1779713119.88059,"t_done":282255.98407392,"t_done_unix":1779713120.9980178},{"request_id":"1363943:13:1438562:865","session_id":"1363943","input_length":28711,"t_proxy_recv":282251.10421011,"t_decision_unix":1779713116.1181512,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9451429765595069,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":27136,"estimated_new_tokens":1575,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":232226,"ongoing_decode_tokens":232226,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1615,"cache_hit":8704,"new_prefill":20007,"score_linear":223522.0,"score_lmetric":40014},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":24064,"new_prefill":4647,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":156900,"ongoing_decode_tokens":29412,"pending_prefill_tokens":5120,"num_requests":2,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":28711,"score_linear":156900.0,"score_lmetric":67662},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":73878,"ongoing_decode_tokens":0,"pending_prefill_tokens":73878,"num_requests":1,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":4096,"new_prefill":24615,"score_linear":69782.0,"score_lmetric":98493},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":239831,"ongoing_decode_tokens":0,"pending_prefill_tokens":131287,"num_requests":3,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":28711,"score_linear":239831.0,"score_lmetric":479994},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":0,"new_prefill":28711,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":27136,"new_prefill":1575,"score_linear":80884.0,"score_lmetric":1575},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":24615,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":80884.0,"chosen_score_lmetric":1575,"t_first_token":282251.458586235,"t_first_token_unix":1779713116.4725301,"t_done":282258.096759457,"t_done_unix":1779713123.1107028},{"request_id":"1314357:9:1435219:861","session_id":"1314357","input_length":106959,"t_proxy_recv":282254.26641145,"t_decision_unix":1779713119.280353,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":106496,"affinity_cache_ratio":0.9956712385119532,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":106496,"estimated_new_tokens":463,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":106496,"new_prefill":463,"score_linear":19451.0,"score_lmetric":463},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":106959,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":156900,"ongoing_decode_tokens":29412,"pending_prefill_tokens":5120,"num_requests":2,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":106959,"score_linear":156900.0,"score_lmetric":224158},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2104,"cache_hit":0,"new_prefill":106959,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131513,"ongoing_decode_tokens":131513,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":106959,"score_linear":131513.0,"score_lmetric":213918},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":52920,"ongoing_decode_tokens":0,"pending_prefill_tokens":3768,"num_requests":1,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":0,"new_prefill":106959,"score_linear":52920.0,"score_lmetric":110727},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":136731,"ongoing_decode_tokens":136731,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":106959,"score_linear":136731.0,"score_lmetric":213918},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":106959,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":19451.0,"chosen_score_lmetric":463,"t_first_token":282254.475124487,"t_first_token_unix":1779713119.489068,"t_done":282258.935271671,"t_done_unix":1779713123.9492154},{"request_id":"1314357:10:1439373:867","session_id":"1314357","input_length":107228,"t_proxy_recv":282258.952437292,"t_decision_unix":1779713123.9663787,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":107008,"affinity_cache_ratio":0.9979482970865818,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":107008,"estimated_new_tokens":220,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1617,"cache_hit":107008,"new_prefill":220,"score_linear":18939.0,"score_lmetric":220},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":107228,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":156900,"ongoing_decode_tokens":29412,"pending_prefill_tokens":5120,"num_requests":2,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":107228,"score_linear":156900.0,"score_lmetric":224696},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2104,"cache_hit":0,"new_prefill":107228,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":64353,"ongoing_decode_tokens":64353,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2181,"cache_hit":0,"new_prefill":107228,"score_linear":64353.0,"score_lmetric":214456},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":0,"new_prefill":107228,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1891,"cache_hit":0,"new_prefill":107228,"score_linear":108020.0,"score_lmetric":107228},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":107228,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":18939.0,"chosen_score_lmetric":220,"t_first_token":282259.118625877,"t_first_token_unix":1779713124.132569,"t_done":282259.658132651,"t_done_unix":1779713124.6720762},{"request_id":"1364090:10:1426859:847","session_id":"1364090","input_length":108020,"t_proxy_recv":282210.508828061,"t_decision_unix":1779713075.5227692,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":101376,"affinity_cache_ratio":0.9384928716904277,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":101376,"estimated_new_tokens":6644,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":7680,"new_prefill":100340,"score_linear":94123.0,"score_lmetric":100340},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":198806,"ongoing_decode_tokens":198806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":4096,"new_prefill":103924,"score_linear":194710.0,"score_lmetric":207848},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":108020,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":42342,"ongoing_decode_tokens":42342,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":7680,"new_prefill":100340,"score_linear":34662.0,"score_lmetric":100340},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":129673,"num_requests":1,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":108020,"score_linear":129673.0,"score_lmetric":237693},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":268537,"ongoing_decode_tokens":195349,"pending_prefill_tokens":4580,"num_requests":4,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":0,"new_prefill":108020,"score_linear":268537.0,"score_lmetric":450400},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":26923,"ongoing_decode_tokens":26923,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":101376,"new_prefill":6644,"score_linear":-74453.0,"score_lmetric":6644},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":7680,"new_prefill":100340,"score_linear":36083.0,"score_lmetric":100340}],"chosen_score_linear":-74453.0,"chosen_score_lmetric":6644,"t_first_token":282214.691724615,"t_first_token_unix":1779713079.7056682,"t_done":282267.911670099,"t_done_unix":1779713132.9256139},{"request_id":"1314357:11:1442554:870","session_id":"1314357","input_length":108166,"t_proxy_recv":282264.315969197,"t_decision_unix":1779713129.3299105,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":107008,"affinity_cache_ratio":0.9892942329382616,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":107008,"estimated_new_tokens":1158,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1617,"cache_hit":107008,"new_prefill":1158,"score_linear":18939.0,"score_lmetric":1158},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":63153,"ongoing_decode_tokens":0,"pending_prefill_tokens":14001,"num_requests":1,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":108166,"score_linear":63153.0,"score_lmetric":122167},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":175956,"ongoing_decode_tokens":29412,"pending_prefill_tokens":10864,"num_requests":3,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":108166,"score_linear":175956.0,"score_lmetric":357090},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2104,"cache_hit":0,"new_prefill":108166,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":64353,"ongoing_decode_tokens":64353,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2181,"cache_hit":0,"new_prefill":108166,"score_linear":64353.0,"score_lmetric":216332},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":0,"new_prefill":108166,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1891,"cache_hit":0,"new_prefill":108166,"score_linear":108020.0,"score_lmetric":108166},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":108166,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":18939.0,"chosen_score_lmetric":1158,"t_first_token":282265.243305771,"t_first_token_unix":1779713130.2572494,"t_done":282268.771978279,"t_done_unix":1779713133.7859216},{"request_id":"1407599:2:1430593:852","session_id":"1407599","input_length":1840,"t_proxy_recv":282223.526747672,"t_decision_unix":1779713088.540689,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":1536,"affinity_cache_ratio":0.8347826086956521,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":304,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":1840,"score_linear":101803.0,"score_lmetric":1840},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":1840,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2368,"cache_hit":0,"new_prefill":1840,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":1840,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":237991,"ongoing_decode_tokens":0,"pending_prefill_tokens":130983,"num_requests":2,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":1536,"new_prefill":304,"score_linear":236455.0,"score_lmetric":262574},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":0,"new_prefill":1840,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1886,"cache_hit":0,"new_prefill":1840,"score_linear":108020.0,"score_lmetric":1840},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":1840,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":236455.0,"chosen_score_lmetric":262574,"t_first_token":282252.248820896,"t_first_token_unix":1779713117.262764,"t_done":282273.814908982,"t_done_unix":1779713138.8288522},{"request_id":"1313181:9:1386945:689","session_id":"1313181","input_length":62513,"t_proxy_recv":282254.295969765,"t_decision_unix":1779713119.3099113,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":62464,"affinity_cache_ratio":0.999216163038088,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":62464,"estimated_new_tokens":49,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":232906,"ongoing_decode_tokens":125947,"pending_prefill_tokens":463,"num_requests":2,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":62513,"score_linear":232906.0,"score_lmetric":125952},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":62513,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":156900,"ongoing_decode_tokens":29412,"pending_prefill_tokens":5120,"num_requests":2,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":12800,"new_prefill":49713,"score_linear":144100.0,"score_lmetric":109666},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2104,"cache_hit":0,"new_prefill":62513,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":1840,"ongoing_decode_tokens":1840,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2181,"cache_hit":62464,"new_prefill":49,"score_linear":-60624.0,"score_lmetric":49},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":52920,"ongoing_decode_tokens":0,"pending_prefill_tokens":3768,"num_requests":1,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":12800,"new_prefill":49713,"score_linear":40120.0,"score_lmetric":53481},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":136731,"ongoing_decode_tokens":136731,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":62513,"score_linear":136731.0,"score_lmetric":125026},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":62464,"new_prefill":49,"score_linear":-62464.0,"score_lmetric":0}],"chosen_score_linear":-60624.0,"chosen_score_lmetric":49,"t_first_token":282254.38750646,"t_first_token_unix":1779713119.40145,"t_done":282274.289223454,"t_done_unix":1779713139.3031666},{"request_id":"1264679:3:1431512:853","session_id":"1264679","input_length":125947,"t_proxy_recv":282226.519815544,"t_decision_unix":1779713091.533757,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":114176,"affinity_cache_ratio":0.9065400525617918,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":114176,"estimated_new_tokens":11771,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":105709,"ongoing_decode_tokens":105709,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1614,"cache_hit":114176,"new_prefill":11771,"score_linear":-8467.0,"score_lmetric":11771},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":125947,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2368,"cache_hit":0,"new_prefill":125947,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":125947,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":239831,"ongoing_decode_tokens":0,"pending_prefill_tokens":131287,"num_requests":3,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":125947,"score_linear":239831.0,"score_lmetric":771702},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":0,"new_prefill":125947,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1886,"cache_hit":0,"new_prefill":125947,"score_linear":108020.0,"score_lmetric":125947},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":125947,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-8467.0,"chosen_score_lmetric":11771,"t_first_token":282235.070311528,"t_first_token_unix":1779713100.084255,"t_done":282277.207139268,"t_done_unix":1779713142.2210827},{"request_id":"1294611:12:1441557:868","session_id":"1294611","input_length":63153,"t_proxy_recv":282260.782171398,"t_decision_unix":1779713125.7961075,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":49152,"affinity_cache_ratio":0.7783003182746663,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49152,"estimated_new_tokens":14001,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1617,"cache_hit":0,"new_prefill":63153,"score_linear":125947.0,"score_lmetric":63153},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":49152,"new_prefill":14001,"score_linear":-49152.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":156900,"ongoing_decode_tokens":29412,"pending_prefill_tokens":5120,"num_requests":2,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":12800,"new_prefill":50353,"score_linear":144100.0,"score_lmetric":110946},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2104,"cache_hit":12800,"new_prefill":50353,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":64353,"ongoing_decode_tokens":64353,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2181,"cache_hit":0,"new_prefill":63153,"score_linear":64353.0,"score_lmetric":126306},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":0,"new_prefill":63153,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1891,"cache_hit":0,"new_prefill":63153,"score_linear":108020.0,"score_lmetric":63153},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":63153,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49152.0,"chosen_score_lmetric":0,"t_first_token":282274.91270168,"t_first_token_unix":1779713139.9266567,"t_done":282277.442879467,"t_done_unix":1779713142.4568226},{"request_id":"1294611:13:1446349:872","session_id":"1294611","input_length":63488,"t_proxy_recv":282277.453317112,"t_decision_unix":1779713142.4672587,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":62976,"affinity_cache_ratio":0.9919354838709677,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":62976,"estimated_new_tokens":512,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":63488,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":62976,"new_prefill":512,"score_linear":-62976.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":175956,"ongoing_decode_tokens":29412,"pending_prefill_tokens":10864,"num_requests":3,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":12800,"new_prefill":50688,"score_linear":163156.0,"score_lmetric":184656},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2104,"cache_hit":12800,"new_prefill":50688,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":2186,"cache_hit":0,"new_prefill":63488,"score_linear":129839.0,"score_lmetric":128815},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":0,"new_prefill":63488,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":63488,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":63488,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-62976.0,"chosen_score_lmetric":0,"t_first_token":282277.561210338,"t_first_token_unix":1779713142.5751538,"t_done":282278.563055694,"t_done_unix":1779713143.5770009},{"request_id":"1314357:12:1446681:873","session_id":"1314357","input_length":108409,"t_proxy_recv":282278.029410451,"t_decision_unix":1779713143.043352,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":108032,"affinity_cache_ratio":0.996522428949626,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":108032,"estimated_new_tokens":377,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":108032,"new_prefill":377,"score_linear":-108032.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":63488,"ongoing_decode_tokens":63488,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":108409,"score_linear":63488.0,"score_lmetric":108409},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":175956,"ongoing_decode_tokens":29412,"pending_prefill_tokens":10864,"num_requests":3,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":108409,"score_linear":175956.0,"score_lmetric":357819},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2104,"cache_hit":0,"new_prefill":108409,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":2186,"cache_hit":0,"new_prefill":108409,"score_linear":129839.0,"score_lmetric":173736},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":0,"new_prefill":108409,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":108409,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":108409,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-108032.0,"chosen_score_lmetric":0,"t_first_token":282278.248014276,"t_first_token_unix":1779713143.261958,"t_done":282278.847360289,"t_done_unix":1779713143.8613038},{"request_id":"1342921:6:1448548:876","session_id":"1342921","input_length":75761,"t_proxy_recv":282284.250565724,"t_decision_unix":1779713149.2645068,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":73728,"affinity_cache_ratio":0.9731656129142963,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73728,"estimated_new_tokens":2033,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":108791,"ongoing_decode_tokens":0,"pending_prefill_tokens":759,"num_requests":1,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":75761,"score_linear":108791.0,"score_lmetric":76520},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1801,"cache_hit":0,"new_prefill":75761,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":175956,"ongoing_decode_tokens":29412,"pending_prefill_tokens":10864,"num_requests":3,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":75761,"score_linear":175956.0,"score_lmetric":259875},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2104,"cache_hit":73728,"new_prefill":2033,"score_linear":-73728.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":2186,"cache_hit":71168,"new_prefill":4593,"score_linear":58671.0,"score_lmetric":69920},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":0,"new_prefill":75761,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":75761,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":75761,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73728.0,"chosen_score_lmetric":0,"t_first_token":282285.218028914,"t_first_token_unix":1779713150.231972,"t_done":282285.581133078,"t_done_unix":1779713150.5950766},{"request_id":"1314357:13:1448346:875","session_id":"1314357","input_length":108791,"t_proxy_recv":282283.696238928,"t_decision_unix":1779713148.71018,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":108032,"affinity_cache_ratio":0.9930233199437454,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":108032,"estimated_new_tokens":759,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":108032,"new_prefill":759,"score_linear":-108032.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1801,"cache_hit":0,"new_prefill":108791,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":175956,"ongoing_decode_tokens":29412,"pending_prefill_tokens":10864,"num_requests":3,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":108791,"score_linear":175956.0,"score_lmetric":358965},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2104,"cache_hit":0,"new_prefill":108791,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":2186,"cache_hit":0,"new_prefill":108791,"score_linear":129839.0,"score_lmetric":174118},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":0,"new_prefill":108791,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":108791,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":108791,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-108032.0,"chosen_score_lmetric":0,"t_first_token":282284.258312189,"t_first_token_unix":1779713149.2722554,"t_done":282286.034356195,"t_done_unix":1779713151.0483},{"request_id":"1294611:14:1449074:877","session_id":"1294611","input_length":63616,"t_proxy_recv":282286.072509606,"t_decision_unix":1779713151.0864513,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":63488,"affinity_cache_ratio":0.9979879275653923,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":63488,"estimated_new_tokens":128,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1644,"cache_hit":0,"new_prefill":63616,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1801,"cache_hit":63488,"new_prefill":128,"score_linear":-63488.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":175956,"ongoing_decode_tokens":29412,"pending_prefill_tokens":10864,"num_requests":3,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":12800,"new_prefill":50816,"score_linear":163156.0,"score_lmetric":185040},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":12800,"new_prefill":50816,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":2186,"cache_hit":0,"new_prefill":63616,"score_linear":129839.0,"score_lmetric":128943},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":0,"new_prefill":63616,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":63616,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":63616,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-63488.0,"chosen_score_lmetric":0,"t_first_token":282286.167721316,"t_first_token_unix":1779713151.1816642,"t_done":282291.029172722,"t_done_unix":1779713156.043116},{"request_id":"1314357:14:1450740:878","session_id":"1314357","input_length":109010,"t_proxy_recv":282291.525150248,"t_decision_unix":1779713156.5390913,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":108544,"affinity_cache_ratio":0.9957251628290983,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":108544,"estimated_new_tokens":466,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1644,"cache_hit":108544,"new_prefill":466,"score_linear":-108544.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":109010,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":175956,"ongoing_decode_tokens":29412,"pending_prefill_tokens":10864,"num_requests":3,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":109010,"score_linear":175956.0,"score_lmetric":359622},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":109010,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":2186,"cache_hit":0,"new_prefill":109010,"score_linear":129839.0,"score_lmetric":174337},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":0,"new_prefill":109010,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":109010,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":109010,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-108544.0,"chosen_score_lmetric":0,"t_first_token":282291.756662514,"t_first_token_unix":1779713156.7706063,"t_done":282293.043984234,"t_done_unix":1779713158.0579278},{"request_id":"1268861:11:1452208:880","session_id":"1268861","input_length":55050,"t_proxy_recv":282296.491231275,"t_decision_unix":1779713161.5051723,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9579654859218892,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":52736,"estimated_new_tokens":2314,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":55050,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":55050,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":175956,"ongoing_decode_tokens":29412,"pending_prefill_tokens":10864,"num_requests":3,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":55050,"score_linear":175956.0,"score_lmetric":197742},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":55050,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":2186,"cache_hit":21504,"new_prefill":33546,"score_linear":108335.0,"score_lmetric":98873},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":52736,"new_prefill":2314,"score_linear":-52736.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":55050,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":55050,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":282297.339086018,"t_first_token_unix":1779713162.3530297,"t_done":282297.816392657,"t_done_unix":1779713162.8303363},{"request_id":"1314357:15:1452292:882","session_id":"1314357","input_length":109164,"t_proxy_recv":282296.750024607,"t_decision_unix":1779713161.7639663,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":109056,"affinity_cache_ratio":0.9990106628558866,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":109056,"estimated_new_tokens":108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":109056,"new_prefill":108,"score_linear":-109056.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":109164,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":175956,"ongoing_decode_tokens":29412,"pending_prefill_tokens":10864,"num_requests":3,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":109164,"score_linear":175956.0,"score_lmetric":360084},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":109164,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":2186,"cache_hit":0,"new_prefill":109164,"score_linear":129839.0,"score_lmetric":174491},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":55050,"ongoing_decode_tokens":0,"pending_prefill_tokens":2314,"num_requests":1,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":0,"new_prefill":109164,"score_linear":55050.0,"score_lmetric":111478},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":109164,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":109164,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-109056.0,"chosen_score_lmetric":0,"t_first_token":282296.922146834,"t_first_token_unix":1779713161.9360902,"t_done":282300.919627986,"t_done_unix":1779713165.9335713},{"request_id":"1371622:2:1442319:869","session_id":"1371622","input_length":19056,"t_proxy_recv":282263.508096874,"t_decision_unix":1779713128.5220382,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":13312,"affinity_cache_ratio":0.6985726280436608,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":13312,"estimated_new_tokens":5744,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1617,"cache_hit":0,"new_prefill":19056,"score_linear":125947.0,"score_lmetric":19056},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":63153,"ongoing_decode_tokens":0,"pending_prefill_tokens":14001,"num_requests":1,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":19056,"score_linear":63153.0,"score_lmetric":33057},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":156900,"ongoing_decode_tokens":29412,"pending_prefill_tokens":5120,"num_requests":2,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":13312,"new_prefill":5744,"score_linear":143588.0,"score_lmetric":21728},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2104,"cache_hit":0,"new_prefill":19056,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":64353,"ongoing_decode_tokens":64353,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2181,"cache_hit":11264,"new_prefill":7792,"score_linear":53089.0,"score_lmetric":15584},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":0,"new_prefill":19056,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1891,"cache_hit":0,"new_prefill":19056,"score_linear":108020.0,"score_lmetric":19056},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":19056,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":143588.0,"chosen_score_lmetric":21728,"t_first_token":282302.747144321,"t_first_token_unix":1779713167.761088,"t_done":282304.20538328,"t_done_unix":1779713169.2193267},{"request_id":"1356885:2:1438037:864","session_id":"1356885","input_length":127488,"t_proxy_recv":282249.396907572,"t_decision_unix":1779713114.4108489,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":122368,"affinity_cache_ratio":0.9598393574297188,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":122368,"estimated_new_tokens":5120,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":232226,"ongoing_decode_tokens":232226,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1615,"cache_hit":0,"new_prefill":127488,"score_linear":232226.0,"score_lmetric":254976},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":127488,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":29412,"ongoing_decode_tokens":29412,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":122368,"new_prefill":5120,"score_linear":-92956.0,"score_lmetric":5120},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":73878,"ongoing_decode_tokens":0,"pending_prefill_tokens":73878,"num_requests":1,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":6656,"new_prefill":120832,"score_linear":67222.0,"score_lmetric":194710},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":239831,"ongoing_decode_tokens":0,"pending_prefill_tokens":131287,"num_requests":3,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":127488,"score_linear":239831.0,"score_lmetric":776325},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":0,"new_prefill":127488,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":127488,"score_linear":108020.0,"score_lmetric":127488},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":127488,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-92956.0,"chosen_score_lmetric":5120,"t_first_token":282301.125375109,"t_first_token_unix":1779713166.1393182,"t_done":282305.04028031,"t_done_unix":1779713170.0542238},{"request_id":"1313181:10:1338837:436","session_id":"1313181","input_length":129839,"t_proxy_recv":282274.310782254,"t_decision_unix":1779713139.3247232,"policy":"unified","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":64512,"affinity_cache_ratio":0.4968614977009989,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":64512,"estimated_new_tokens":65327,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1619,"cache_hit":0,"new_prefill":129839,"score_linear":125947.0,"score_lmetric":129839},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":63153,"ongoing_decode_tokens":0,"pending_prefill_tokens":14001,"num_requests":1,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":129839,"score_linear":63153.0,"score_lmetric":143840},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":175956,"ongoing_decode_tokens":29412,"pending_prefill_tokens":10864,"num_requests":3,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":12800,"new_prefill":117039,"score_linear":163156.0,"score_lmetric":383709},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2104,"cache_hit":0,"new_prefill":129839,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2186,"cache_hit":64512,"new_prefill":65327,"score_linear":-64512.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":12800,"new_prefill":117039,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":129839,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":62464,"new_prefill":67375,"score_linear":-62464.0,"score_lmetric":0}],"chosen_score_linear":-64512.0,"chosen_score_lmetric":0,"t_first_token":282313.345142301,"t_first_token_unix":1779713178.3590858,"t_done":282314.226171054,"t_done_unix":1779713179.2401145},{"request_id":"1313181:10:1399867:746","session_id":"1313181","input_length":64724,"t_proxy_recv":282314.237784784,"t_decision_unix":1779713179.2517264,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":64512,"affinity_cache_ratio":0.9967245534886595,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":64512,"estimated_new_tokens":212,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":64724,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":64724,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":177267,"ongoing_decode_tokens":177267,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":2392,"cache_hit":12800,"new_prefill":51924,"score_linear":164467.0,"score_lmetric":155772},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":64724,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2187,"cache_hit":64512,"new_prefill":212,"score_linear":-64512.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2182,"cache_hit":12800,"new_prefill":51924,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":64724,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":62464,"new_prefill":2260,"score_linear":-62464.0,"score_lmetric":0}],"chosen_score_linear":-64512.0,"chosen_score_lmetric":0,"t_first_token":282314.327629347,"t_first_token_unix":1779713179.3415728,"t_done":282317.46597047,"t_done_unix":1779713182.4799137},{"request_id":"1363943:14:1458616:885","session_id":"1363943","input_length":29268,"t_proxy_recv":282318.226048383,"t_decision_unix":1779713183.2399895,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":29184,"affinity_cache_ratio":0.997129971299713,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":29184,"estimated_new_tokens":84,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":8704,"new_prefill":20564,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":5204,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":177267,"ongoing_decode_tokens":177267,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":2392,"cache_hit":0,"new_prefill":29268,"score_linear":177267.0,"score_lmetric":87804},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":4096,"new_prefill":25172,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":2188,"cache_hit":0,"new_prefill":29268,"score_linear":129931.0,"score_lmetric":94175},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2182,"cache_hit":0,"new_prefill":29268,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":29184,"new_prefill":84,"score_linear":-29184.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":25172,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-29184.0,"chosen_score_lmetric":0,"t_first_token":282318.308885854,"t_first_token_unix":1779713183.3228292,"t_done":282318.873799366,"t_done_unix":1779713183.8877425},{"request_id":"1371622:3:1443876:871","session_id":"1371622","input_length":19292,"t_proxy_recv":282304.211193218,"t_decision_unix":1779713169.2251346,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":18944,"affinity_cache_ratio":0.9819614347916235,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":18944,"estimated_new_tokens":348,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":19292,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":19292,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":156900,"ongoing_decode_tokens":156900,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2382,"cache_hit":18944,"new_prefill":348,"score_linear":137956.0,"score_lmetric":696},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":19292,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":2186,"cache_hit":11264,"new_prefill":8028,"score_linear":118575.0,"score_lmetric":73355},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2182,"cache_hit":0,"new_prefill":19292,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":19292,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":19292,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":137956.0,"chosen_score_lmetric":696,"t_first_token":282304.314506977,"t_first_token_unix":1779713169.3284504,"t_done":282323.222556325,"t_done_unix":1779713188.2365017},{"request_id":"1371622:4:1452210:881","session_id":"1371622","input_length":20439,"t_proxy_recv":282323.22806824,"t_decision_unix":1779713188.2420099,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":19968,"affinity_cache_ratio":0.9769558197563482,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":471,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":26406,"ongoing_decode_tokens":0,"pending_prefill_tokens":294,"num_requests":1,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":20439,"score_linear":26406.0,"score_lmetric":20733},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":20439,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":157975,"ongoing_decode_tokens":157975,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2394,"cache_hit":19968,"new_prefill":471,"score_linear":138007.0,"score_lmetric":942},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":20439,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":2188,"cache_hit":11264,"new_prefill":9175,"score_linear":118667.0,"score_lmetric":74082},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2182,"cache_hit":0,"new_prefill":20439,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":20439,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":20439,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":138007.0,"chosen_score_lmetric":942,"t_first_token":282323.331770799,"t_first_token_unix":1779713188.3457146,"t_done":282324.955261693,"t_done_unix":1779713189.969205},{"request_id":"1371622:5:1455608:884","session_id":"1371622","input_length":21518,"t_proxy_recv":282324.962352826,"t_decision_unix":1779713189.9762936,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":20480,"affinity_cache_ratio":0.951761316107445,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":20480,"estimated_new_tokens":1038,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":26406,"ongoing_decode_tokens":0,"pending_prefill_tokens":294,"num_requests":1,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":21518,"score_linear":26406.0,"score_lmetric":21812},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":21518,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":157975,"ongoing_decode_tokens":157975,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2395,"cache_hit":20480,"new_prefill":1038,"score_linear":137495.0,"score_lmetric":2076},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":21518,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":2188,"cache_hit":11264,"new_prefill":10254,"score_linear":118667.0,"score_lmetric":75161},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2182,"cache_hit":0,"new_prefill":21518,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":21518,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":21518,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":137495.0,"chosen_score_lmetric":2076,"t_first_token":282325.296591044,"t_first_token_unix":1779713190.3105342,"t_done":282326.865660526,"t_done_unix":1779713191.8796036},{"request_id":"1242838:5:1460026:886","session_id":"1242838","input_length":26406,"t_proxy_recv":282322.708704651,"t_decision_unix":1779713187.7226458,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":26112,"affinity_cache_ratio":0.9888661667802772,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":26112,"estimated_new_tokens":294,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":26112,"new_prefill":294,"score_linear":-26112.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":26406,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":177267,"ongoing_decode_tokens":177267,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":2392,"cache_hit":3584,"new_prefill":22822,"score_linear":173683.0,"score_lmetric":68466},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":26406,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":2188,"cache_hit":0,"new_prefill":26406,"score_linear":129931.0,"score_lmetric":91313},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2182,"cache_hit":3584,"new_prefill":22822,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":26406,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":26406,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-26112.0,"chosen_score_lmetric":0,"t_first_token":282325.899139478,"t_first_token_unix":1779713190.9130828,"t_done":282327.840919572,"t_done_unix":1779713192.8548634},{"request_id":"1356885:3:1451307:879","session_id":"1356885","input_length":128563,"t_proxy_recv":282305.062925994,"t_decision_unix":1779713170.0768673,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":127488,"affinity_cache_ratio":0.9916383407356705,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":127488,"estimated_new_tokens":1075,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":128563,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":128563,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":48704,"ongoing_decode_tokens":48704,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2392,"cache_hit":127488,"new_prefill":1075,"score_linear":-78784.0,"score_lmetric":2150},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":6656,"new_prefill":121907,"score_linear":-6656.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":2186,"cache_hit":0,"new_prefill":128563,"score_linear":129839.0,"score_lmetric":193890},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2182,"cache_hit":0,"new_prefill":128563,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":128563,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":128563,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-78784.0,"chosen_score_lmetric":2150,"t_first_token":282306.151134396,"t_first_token_unix":1779713171.1650782,"t_done":282329.509752854,"t_done_unix":1779713194.5236971},{"request_id":"1363943:15:1461716:889","session_id":"1363943","input_length":29651,"t_proxy_recv":282328.144430941,"t_decision_unix":1779713193.158372,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":29184,"affinity_cache_ratio":0.984250109608445,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":29184,"estimated_new_tokens":467,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":8704,"new_prefill":20947,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":5587,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":157975,"ongoing_decode_tokens":157975,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2397,"cache_hit":0,"new_prefill":29651,"score_linear":157975.0,"score_lmetric":59302},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":4096,"new_prefill":25555,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":2188,"cache_hit":0,"new_prefill":29651,"score_linear":129931.0,"score_lmetric":94558},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":81299,"ongoing_decode_tokens":0,"pending_prefill_tokens":8083,"num_requests":1,"active_p_offloads":0,"cached_blocks":2182,"cache_hit":0,"new_prefill":29651,"score_linear":81299.0,"score_lmetric":37734},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":29184,"new_prefill":467,"score_linear":-29184.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":25555,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-29184.0,"chosen_score_lmetric":0,"t_first_token":282328.317843664,"t_first_token_unix":1779713193.3317873,"t_done":282329.699848637,"t_done_unix":1779713194.7137918},{"request_id":"1363093:6:1460799:888","session_id":"1363093","input_length":81299,"t_proxy_recv":282325.288953538,"t_decision_unix":1779713190.3028946,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":73216,"affinity_cache_ratio":0.9005768828644879,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73216,"estimated_new_tokens":8083,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":26406,"ongoing_decode_tokens":0,"pending_prefill_tokens":294,"num_requests":1,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":81299,"score_linear":26406.0,"score_lmetric":81593},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":81299,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":179493,"ongoing_decode_tokens":157975,"pending_prefill_tokens":1038,"num_requests":3,"active_p_offloads":0,"cached_blocks":2395,"cache_hit":0,"new_prefill":81299,"score_linear":179493.0,"score_lmetric":247011},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":81299,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":2188,"cache_hit":0,"new_prefill":81299,"score_linear":129931.0,"score_lmetric":146206},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2182,"cache_hit":73216,"new_prefill":8083,"score_linear":-73216.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":81299,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":81299,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73216.0,"chosen_score_lmetric":0,"t_first_token":282329.211423143,"t_first_token_unix":1779713194.225367,"t_done":282329.826196904,"t_done_unix":1779713194.84014},{"request_id":"1404395:2:1435941:862","session_id":"1404395","input_length":29412,"t_proxy_recv":282242.290288945,"t_decision_unix":1779713107.30423,"policy":"unified","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":2,"affinity_cache_hit":22528,"affinity_cache_ratio":0.7659458724330205,"affinity_num_requests":3,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":29412,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":231656,"ongoing_decode_tokens":231656,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1614,"cache_hit":512,"new_prefill":28900,"score_linear":231144.0,"score_lmetric":57800},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1773,"cache_hit":0,"new_prefill":29412,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":29412,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":73878,"ongoing_decode_tokens":0,"pending_prefill_tokens":73878,"num_requests":1,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":29412,"score_linear":73878.0,"score_lmetric":103290},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":239831,"ongoing_decode_tokens":0,"pending_prefill_tokens":131287,"num_requests":3,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":22528,"new_prefill":6884,"score_linear":217303.0,"score_lmetric":414513},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":0,"new_prefill":29412,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":29412,"score_linear":108020.0,"score_lmetric":29412},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":29412,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282246.086408494,"t_first_token_unix":1779713111.1003523,"t_done":282333.888992049,"t_done_unix":1779713198.9029353},{"request_id":"1404395:3:1460072:887","session_id":"1404395","input_length":34392,"t_proxy_recv":282333.898483637,"t_decision_unix":1779713198.912425,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":31744,"affinity_cache_ratio":0.9230053500814143,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":31744,"estimated_new_tokens":2648,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":512,"new_prefill":33880,"score_linear":-512.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":34392,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2459,"cache_hit":31744,"new_prefill":2648,"score_linear":-31744.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":34392,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":2188,"cache_hit":22528,"new_prefill":11864,"score_linear":107403.0,"score_lmetric":76771},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2197,"cache_hit":0,"new_prefill":34392,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":0,"new_prefill":34392,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":34392,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-31744.0,"chosen_score_lmetric":0,"t_first_token":282334.51743055,"t_first_token_unix":1779713199.5313747,"t_done":282336.947213085,"t_done_unix":1779713201.9611566},{"request_id":"1363943:16:1463626:890","session_id":"1363943","input_length":34671,"t_proxy_recv":282334.613849352,"t_decision_unix":1779713199.62779,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":29696,"affinity_cache_ratio":0.8565083210752502,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":29696,"estimated_new_tokens":4975,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":8704,"new_prefill":25967,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":10607,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":34392,"ongoing_decode_tokens":34392,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2459,"cache_hit":0,"new_prefill":34671,"score_linear":34392.0,"score_lmetric":34671},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":4096,"new_prefill":30575,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":2188,"cache_hit":0,"new_prefill":34671,"score_linear":129931.0,"score_lmetric":99578},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2197,"cache_hit":0,"new_prefill":34671,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":29696,"new_prefill":4975,"score_linear":-29696.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":30575,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-29696.0,"chosen_score_lmetric":0,"t_first_token":282335.771616147,"t_first_token_unix":1779713200.78556,"t_done":282337.239478621,"t_done_unix":1779713202.2534218},{"request_id":"1268861:12:1463977:891","session_id":"1268861","input_length":56147,"t_proxy_recv":282335.900380474,"t_decision_unix":1779713200.914322,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9757244376369174,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54784,"estimated_new_tokens":1363,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":56147,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":56147,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":34392,"ongoing_decode_tokens":34392,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2459,"cache_hit":0,"new_prefill":56147,"score_linear":34392.0,"score_lmetric":56147},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":56147,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":2188,"cache_hit":21504,"new_prefill":34643,"score_linear":108427.0,"score_lmetric":99550},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2197,"cache_hit":54784,"new_prefill":1363,"score_linear":-54784.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":34671,"ongoing_decode_tokens":34671,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":0,"new_prefill":56147,"score_linear":34671.0,"score_lmetric":56147},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":56147,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":282336.422940221,"t_first_token_unix":1779713201.4368832,"t_done":282337.325696177,"t_done_unix":1779713202.339639},{"request_id":"1371622:6:1465333:892","session_id":"1371622","input_length":23079,"t_proxy_recv":282340.159742922,"t_decision_unix":1779713205.173684,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":21504,"affinity_cache_ratio":0.9317561419472248,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":21504,"estimated_new_tokens":1575,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":23079,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":23079,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2464,"cache_hit":21504,"new_prefill":1575,"score_linear":-21504.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":23079,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":2188,"cache_hit":11264,"new_prefill":11815,"score_linear":118667.0,"score_lmetric":76722},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2199,"cache_hit":0,"new_prefill":23079,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":0,"new_prefill":23079,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":23079,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-21504.0,"chosen_score_lmetric":0,"t_first_token":282340.471175373,"t_first_token_unix":1779713205.4851193,"t_done":282340.9933134,"t_done_unix":1779713206.007257},{"request_id":"1363943:17:1466811:893","session_id":"1363943","input_length":35991,"t_proxy_recv":282345.296350382,"t_decision_unix":1779713210.3102915,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9673529493484482,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":1175,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":8704,"new_prefill":27287,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":11927,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2467,"cache_hit":0,"new_prefill":35991,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":4096,"new_prefill":31895,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":2188,"cache_hit":0,"new_prefill":35991,"score_linear":129931.0,"score_lmetric":100898},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2199,"cache_hit":0,"new_prefill":35991,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":34816,"new_prefill":1175,"score_linear":-34816.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":31895,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-34816.0,"chosen_score_lmetric":0,"t_first_token":282345.657168019,"t_first_token_unix":1779713210.671111,"t_done":282346.782637255,"t_done_unix":1779713211.7965806},{"request_id":"1404395:4:1467473:894","session_id":"1404395","input_length":34863,"t_proxy_recv":282347.610273735,"t_decision_unix":1779713212.6242144,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9839658090238935,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34304,"estimated_new_tokens":559,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":512,"new_prefill":34351,"score_linear":-512.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":34863,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2467,"cache_hit":34304,"new_prefill":559,"score_linear":-34304.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":34863,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":2188,"cache_hit":22528,"new_prefill":12335,"score_linear":107403.0,"score_lmetric":77242},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2199,"cache_hit":0,"new_prefill":34863,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1918,"cache_hit":0,"new_prefill":34863,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":34863,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-34304.0,"chosen_score_lmetric":0,"t_first_token":282347.717274918,"t_first_token_unix":1779713212.7312183,"t_done":282348.967586252,"t_done_unix":1779713213.9815304},{"request_id":"1371622:7:1468305:895","session_id":"1371622","input_length":24036,"t_proxy_recv":282350.3821149,"t_decision_unix":1779713215.3960562,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9585621567648527,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":23040,"estimated_new_tokens":996,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":24036,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":24036,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2468,"cache_hit":23040,"new_prefill":996,"score_linear":-23040.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":24036,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":2188,"cache_hit":11264,"new_prefill":12772,"score_linear":118667.0,"score_lmetric":77679},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2199,"cache_hit":0,"new_prefill":24036,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1918,"cache_hit":0,"new_prefill":24036,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":24036,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-23040.0,"chosen_score_lmetric":0,"t_first_token":282350.604449608,"t_first_token_unix":1779713215.6183937,"t_done":282351.451151862,"t_done_unix":1779713216.4650953},{"request_id":"1313181:11:1344301:464","session_id":"1313181","input_length":129931,"t_proxy_recv":282317.494866477,"t_decision_unix":1779713182.508808,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":65024,"affinity_cache_ratio":0.5004502389729933,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":65024,"estimated_new_tokens":64907,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":129931,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":129931,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":177267,"ongoing_decode_tokens":177267,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":2392,"cache_hit":12800,"new_prefill":117131,"score_linear":164467.0,"score_lmetric":351393},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":129931,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2188,"cache_hit":65024,"new_prefill":64907,"score_linear":-65024.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2182,"cache_hit":12800,"new_prefill":117131,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":129931,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":62464,"new_prefill":67467,"score_linear":-62464.0,"score_lmetric":0}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":282356.471327284,"t_first_token_unix":1779713221.4852712,"t_done":282357.318316753,"t_done_unix":1779713222.3322604},{"request_id":"1313181:11:1402841:762","session_id":"1313181","input_length":65123,"t_proxy_recv":282357.329853372,"t_decision_unix":1779713222.3437946,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9984797997635244,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":65024,"estimated_new_tokens":99,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":65123,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":65123,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2470,"cache_hit":12800,"new_prefill":52323,"score_linear":-12800.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":65123,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":65024,"new_prefill":99,"score_linear":-65024.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2199,"cache_hit":12800,"new_prefill":52323,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":113549,"ongoing_decode_tokens":47303,"pending_prefill_tokens":25798,"num_requests":2,"active_p_offloads":0,"cached_blocks":1918,"cache_hit":0,"new_prefill":65123,"score_linear":113549.0,"score_lmetric":181842},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":62464,"new_prefill":2659,"score_linear":-62464.0,"score_lmetric":0}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":282357.434294608,"t_first_token_unix":1779713222.4482381,"t_done":282357.816554081,"t_done_unix":1779713222.8304975},{"request_id":"1399948:13:1472968:898","session_id":"1399948","input_length":12247,"t_proxy_recv":282365.730744721,"t_decision_unix":1779713230.744686,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":11776,"affinity_cache_ratio":0.9615416020249857,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":471,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":12247,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":12247,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2470,"cache_hit":0,"new_prefill":12247,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":12247,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":65455,"num_requests":1,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":0,"new_prefill":12247,"score_linear":130479.0,"score_lmetric":77702},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2199,"cache_hit":0,"new_prefill":12247,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":113549,"ongoing_decode_tokens":113549,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1918,"cache_hit":11776,"new_prefill":471,"score_linear":101773.0,"score_lmetric":942},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":12247,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":101773.0,"chosen_score_lmetric":942,"t_first_token":282365.804326916,"t_first_token_unix":1779713230.8182702,"t_done":282366.149743167,"t_done_unix":1779713231.1636865},{"request_id":"1405155:3:1469167:897","session_id":"1405155","input_length":66246,"t_proxy_recv":282353.27759542,"t_decision_unix":1779713218.2915366,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":40448,"affinity_cache_ratio":0.610572713824231,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":25798,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":66246,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":512,"new_prefill":65734,"score_linear":-512.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2470,"cache_hit":0,"new_prefill":66246,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":66246,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":2188,"cache_hit":0,"new_prefill":66246,"score_linear":129931.0,"score_lmetric":131153},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2199,"cache_hit":0,"new_prefill":66246,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47303,"ongoing_decode_tokens":0,"pending_prefill_tokens":11463,"num_requests":1,"active_p_offloads":0,"cached_blocks":1918,"cache_hit":40448,"new_prefill":25798,"score_linear":6855.0,"score_lmetric":37261},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":66246,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":6855.0,"chosen_score_lmetric":37261,"t_first_token":282365.323746321,"t_first_token_unix":1779713230.3376899,"t_done":282367.89775014,"t_done_unix":1779713232.9116936},{"request_id":"1363943:18:1469143:896","session_id":"1363943","input_length":47303,"t_proxy_recv":282353.199787172,"t_decision_unix":1779713218.2137284,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":35840,"affinity_cache_ratio":0.7576686468088705,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35840,"estimated_new_tokens":11463,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":8704,"new_prefill":38599,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":23239,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2470,"cache_hit":0,"new_prefill":47303,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":4096,"new_prefill":43207,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":2188,"cache_hit":0,"new_prefill":47303,"score_linear":129931.0,"score_lmetric":112210},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2199,"cache_hit":0,"new_prefill":47303,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1918,"cache_hit":35840,"new_prefill":11463,"score_linear":-35840.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":43207,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-35840.0,"chosen_score_lmetric":0,"t_first_token":282356.456342807,"t_first_token_unix":1779713221.4702868,"t_done":282369.00371137,"t_done_unix":1779713234.017655},{"request_id":"1342921:7:1473812:900","session_id":"1342921","input_length":76600,"t_proxy_recv":282368.695417599,"t_decision_unix":1779713233.709359,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":75776,"affinity_cache_ratio":0.9892428198433421,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75776,"estimated_new_tokens":824,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":76600,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":76600,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":24569,"ongoing_decode_tokens":24569,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2470,"cache_hit":0,"new_prefill":76600,"score_linear":24569.0,"score_lmetric":76600},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":75776,"new_prefill":824,"score_linear":-75776.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":65455,"num_requests":1,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":71168,"new_prefill":5432,"score_linear":59311.0,"score_lmetric":70887},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2199,"cache_hit":0,"new_prefill":76600,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47303,"ongoing_decode_tokens":47303,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1968,"cache_hit":0,"new_prefill":76600,"score_linear":47303.0,"score_lmetric":76600},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":76600,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75776.0,"chosen_score_lmetric":0,"t_first_token":282369.23653431,"t_first_token_unix":1779713234.2504778,"t_done":282370.57520356,"t_done_unix":1779713235.5891473},{"request_id":"1371622:8:1473416:899","session_id":"1371622","input_length":24569,"t_proxy_recv":282367.400034204,"t_decision_unix":1779713232.4139755,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9794456428833083,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":505,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":24569,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":24569,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2470,"cache_hit":24064,"new_prefill":505,"score_linear":-24064.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":24569,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":65455,"num_requests":1,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":11264,"new_prefill":13305,"score_linear":119215.0,"score_lmetric":78760},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2199,"cache_hit":0,"new_prefill":24569,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":113549,"ongoing_decode_tokens":113549,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1918,"cache_hit":0,"new_prefill":24569,"score_linear":113549.0,"score_lmetric":49138},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":24569,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":282367.557512298,"t_first_token_unix":1779713232.5714555,"t_done":282370.716019078,"t_done_unix":1779713235.7299623},{"request_id":"1363943:19:1473899:901","session_id":"1363943","input_length":56056,"t_proxy_recv":282369.021947848,"t_decision_unix":1779713234.0358891,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":47616,"affinity_cache_ratio":0.8494362780077066,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":47616,"estimated_new_tokens":8440,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":8704,"new_prefill":47352,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":31992,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":24569,"ongoing_decode_tokens":24569,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2470,"cache_hit":0,"new_prefill":56056,"score_linear":24569.0,"score_lmetric":56056},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76600,"ongoing_decode_tokens":0,"pending_prefill_tokens":824,"num_requests":1,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":4096,"new_prefill":51960,"score_linear":72504.0,"score_lmetric":52784},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":65455,"num_requests":1,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":0,"new_prefill":56056,"score_linear":130479.0,"score_lmetric":121511},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2199,"cache_hit":0,"new_prefill":56056,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1991,"cache_hit":47616,"new_prefill":8440,"score_linear":-47616.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":51960,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-47616.0,"chosen_score_lmetric":0,"t_first_token":282371.966474839,"t_first_token_unix":1779713236.9804184,"t_done":282373.828240956,"t_done_unix":1779713238.842186},{"request_id":"1363943:20:1476805:903","session_id":"1363943","input_length":60761,"t_proxy_recv":282378.942665736,"t_decision_unix":1779713243.9566069,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":55808,"affinity_cache_ratio":0.9184838959200803,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":55808,"estimated_new_tokens":4953,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":8704,"new_prefill":52057,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":36697,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2471,"cache_hit":0,"new_prefill":60761,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":4096,"new_prefill":56665,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":65455,"num_requests":1,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":0,"new_prefill":60761,"score_linear":130479.0,"score_lmetric":126216},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44949,"ongoing_decode_tokens":0,"pending_prefill_tokens":44949,"num_requests":1,"active_p_offloads":0,"cached_blocks":2199,"cache_hit":0,"new_prefill":60761,"score_linear":44949.0,"score_lmetric":105710},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2007,"cache_hit":55808,"new_prefill":4953,"score_linear":-55808.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":56665,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-55808.0,"chosen_score_lmetric":0,"t_first_token":282380.729550245,"t_first_token_unix":1779713245.743494,"t_done":282384.186399261,"t_done_unix":1779713249.2003424},{"request_id":"1305906:4:1476757:902","session_id":"1305906","input_length":44949,"t_proxy_recv":282378.762849498,"t_decision_unix":1779713243.7767906,"policy":"unified","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":5,"affinity_cache_hit":16896,"affinity_cache_ratio":0.375892678368818,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":44949,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2471,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":65455,"num_requests":1,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":16896,"new_prefill":28053,"score_linear":113583.0,"score_lmetric":93508},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2199,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2007,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282386.51390213,"t_first_token_unix":1779713251.5278456,"t_done":282387.849101498,"t_done_unix":1779713252.863045},{"request_id":"1371622:9:1479535:905","session_id":"1371622","input_length":25286,"t_proxy_recv":282388.077886242,"t_decision_unix":1779713253.0918276,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":24576,"affinity_cache_ratio":0.9719212212291386,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24576,"estimated_new_tokens":710,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":25286,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":25286,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38827,"ongoing_decode_tokens":0,"pending_prefill_tokens":427,"num_requests":1,"active_p_offloads":0,"cached_blocks":2471,"cache_hit":24576,"new_prefill":710,"score_linear":14251.0,"score_lmetric":1137},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":0,"new_prefill":25286,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":65455,"num_requests":1,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":11264,"new_prefill":14022,"score_linear":119215.0,"score_lmetric":79477},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2287,"cache_hit":0,"new_prefill":25286,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2017,"cache_hit":0,"new_prefill":25286,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":25286,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":14251.0,"chosen_score_lmetric":1137,"t_first_token":282390.830753869,"t_first_token_unix":1779713255.8446972,"t_done":282392.382895974,"t_done_unix":1779713257.3968394},{"request_id":"1404395:5:1480170:906","session_id":"1404395","input_length":36988,"t_proxy_recv":282390.320261437,"t_decision_unix":1779713255.3342023,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9412782524061858,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":2172,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":512,"new_prefill":36476,"score_linear":-512.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":36988,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":64113,"ongoing_decode_tokens":0,"pending_prefill_tokens":1137,"num_requests":2,"active_p_offloads":0,"cached_blocks":2471,"cache_hit":34816,"new_prefill":2172,"score_linear":29297.0,"score_lmetric":6618},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":0,"new_prefill":36988,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":65455,"num_requests":1,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":22528,"new_prefill":14460,"score_linear":107951.0,"score_lmetric":79915},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2287,"cache_hit":0,"new_prefill":36988,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2017,"cache_hit":0,"new_prefill":36988,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":36988,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":29297.0,"chosen_score_lmetric":6618,"t_first_token":282391.368274001,"t_first_token_unix":1779713256.3822181,"t_done":282395.246280908,"t_done_unix":1779713260.2602243},{"request_id":"1268861:13:1481831:907","session_id":"1268861","input_length":57130,"t_proxy_recv":282395.910356574,"t_decision_unix":1779713260.9242978,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":55808,"affinity_cache_ratio":0.976859793453527,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":55808,"estimated_new_tokens":1322,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":57130,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":57130,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38827,"ongoing_decode_tokens":38827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2476,"cache_hit":0,"new_prefill":57130,"score_linear":38827.0,"score_lmetric":57130},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":0,"new_prefill":57130,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":65455,"num_requests":1,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":21504,"new_prefill":35626,"score_linear":108975.0,"score_lmetric":101081},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2287,"cache_hit":55808,"new_prefill":1322,"score_linear":-55808.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2017,"cache_hit":0,"new_prefill":57130,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":57130,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-55808.0,"chosen_score_lmetric":0,"t_first_token":282396.353925422,"t_first_token_unix":1779713261.3678691,"t_done":282397.645214701,"t_done_unix":1779713262.659158},{"request_id":"1313181:12:1346194:470","session_id":"1313181","input_length":130479,"t_proxy_recv":282357.838084404,"t_decision_unix":1779713222.8520257,"policy":"unified","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":65024,"affinity_cache_ratio":0.49834839322802904,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":65024,"estimated_new_tokens":65455,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":130479,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":130479,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2470,"cache_hit":12800,"new_prefill":117679,"score_linear":-12800.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2108,"cache_hit":0,"new_prefill":130479,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":65024,"new_prefill":65455,"score_linear":-65024.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2199,"cache_hit":12800,"new_prefill":117679,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":113549,"ongoing_decode_tokens":47303,"pending_prefill_tokens":25798,"num_requests":2,"active_p_offloads":0,"cached_blocks":1918,"cache_hit":0,"new_prefill":130479,"score_linear":113549.0,"score_lmetric":312554},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":62464,"new_prefill":68015,"score_linear":-62464.0,"score_lmetric":0}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":282397.098600543,"t_first_token_unix":1779713262.1125443,"t_done":282398.003699802,"t_done_unix":1779713263.0176435},{"request_id":"1313181:12:1404813:772","session_id":"1313181","input_length":65179,"t_proxy_recv":282398.014717755,"t_decision_unix":1779713263.0286589,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9976219334448212,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":65024,"estimated_new_tokens":155,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":65179,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":65179,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38827,"ongoing_decode_tokens":38827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2476,"cache_hit":12800,"new_prefill":52379,"score_linear":26027.0,"score_lmetric":52379},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":0,"new_prefill":65179,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2192,"cache_hit":65024,"new_prefill":155,"score_linear":-65024.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2289,"cache_hit":12800,"new_prefill":52379,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":65658,"ongoing_decode_tokens":0,"pending_prefill_tokens":4730,"num_requests":1,"active_p_offloads":0,"cached_blocks":2017,"cache_hit":0,"new_prefill":65179,"score_linear":65658.0,"score_lmetric":69909},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":62464,"new_prefill":2715,"score_linear":-62464.0,"score_lmetric":0}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":282398.113330884,"t_first_token_unix":1779713263.1272743,"t_done":282398.662945541,"t_done_unix":1779713263.676889},{"request_id":"1363943:21:1482184:908","session_id":"1363943","input_length":65658,"t_proxy_recv":282397.244854204,"t_decision_unix":1779713262.2587955,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":60928,"affinity_cache_ratio":0.9279600353346127,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":60928,"estimated_new_tokens":4730,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":8704,"new_prefill":56954,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":41594,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38827,"ongoing_decode_tokens":38827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2476,"cache_hit":0,"new_prefill":65658,"score_linear":38827.0,"score_lmetric":65658},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":4096,"new_prefill":61562,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":130479,"ongoing_decode_tokens":130479,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":0,"new_prefill":65658,"score_linear":130479.0,"score_lmetric":65658},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":57130,"ongoing_decode_tokens":57130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2287,"cache_hit":0,"new_prefill":65658,"score_linear":57130.0,"score_lmetric":65658},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2017,"cache_hit":60928,"new_prefill":4730,"score_linear":-60928.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":61562,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-60928.0,"chosen_score_lmetric":0,"t_first_token":282399.180714834,"t_first_token_unix":1779713264.1946583,"t_done":282402.320227315,"t_done_unix":1779713267.3341708},{"request_id":"1301929:4:1479037:904","session_id":"1301929","input_length":38827,"t_proxy_recv":282386.336461859,"t_decision_unix":1779713251.350403,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":38400,"affinity_cache_ratio":0.9890024982615191,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":38400,"estimated_new_tokens":427,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":3584,"new_prefill":35243,"score_linear":-3584.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":38827,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2471,"cache_hit":38400,"new_prefill":427,"score_linear":-38400.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":0,"new_prefill":38827,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":65455,"num_requests":1,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":0,"new_prefill":38827,"score_linear":130479.0,"score_lmetric":104282},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44949,"ongoing_decode_tokens":0,"pending_prefill_tokens":44949,"num_requests":1,"active_p_offloads":0,"cached_blocks":2199,"cache_hit":38400,"new_prefill":427,"score_linear":6549.0,"score_lmetric":45376},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2017,"cache_hit":0,"new_prefill":38827,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":38827,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-38400.0,"chosen_score_lmetric":0,"t_first_token":282390.764882171,"t_first_token_unix":1779713255.778826,"t_done":282404.60677157,"t_done_unix":1779713269.6207159},{"request_id":"1404395:6:1483967:909","session_id":"1404395","input_length":37689,"t_proxy_recv":282403.16188149,"t_decision_unix":1779713268.1758223,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":36864,"affinity_cache_ratio":0.9781103239672053,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":36864,"estimated_new_tokens":825,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":512,"new_prefill":37177,"score_linear":-512.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":37689,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38827,"ongoing_decode_tokens":38827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2476,"cache_hit":36864,"new_prefill":825,"score_linear":1963.0,"score_lmetric":825},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":0,"new_prefill":37689,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":2192,"cache_hit":22528,"new_prefill":15161,"score_linear":109171.0,"score_lmetric":81836},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2289,"cache_hit":0,"new_prefill":37689,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2026,"cache_hit":0,"new_prefill":37689,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":37689,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":1963.0,"chosen_score_lmetric":825,"t_first_token":282403.374830477,"t_first_token_unix":1779713268.3887742,"t_done":282406.235659577,"t_done_unix":1779713271.249603},{"request_id":"1363943:22:1485341:910","session_id":"1363943","input_length":70238,"t_proxy_recv":282407.897336918,"t_decision_unix":1779713272.9112782,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":65536,"affinity_cache_ratio":0.9330561804151599,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":65536,"estimated_new_tokens":4702,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":8704,"new_prefill":61534,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":46174,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2478,"cache_hit":0,"new_prefill":70238,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":4096,"new_prefill":66142,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":2192,"cache_hit":0,"new_prefill":70238,"score_linear":131699.0,"score_lmetric":136913},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2289,"cache_hit":0,"new_prefill":70238,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2026,"cache_hit":65536,"new_prefill":4702,"score_linear":-65536.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":66142,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-65536.0,"chosen_score_lmetric":0,"t_first_token":282409.827330637,"t_first_token_unix":1779713274.8412743,"t_done":282413.306458709,"t_done_unix":1779713278.3204021},{"request_id":"1242838:6:1486863:911","session_id":"1242838","input_length":29903,"t_proxy_recv":282412.913328244,"t_decision_unix":1779713277.9272697,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":26624,"affinity_cache_ratio":0.8903454502892686,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":26624,"estimated_new_tokens":3279,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":26624,"new_prefill":3279,"score_linear":-26624.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":29903,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2478,"cache_hit":3584,"new_prefill":26319,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":0,"new_prefill":29903,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":2192,"cache_hit":0,"new_prefill":29903,"score_linear":131699.0,"score_lmetric":96578},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2289,"cache_hit":3584,"new_prefill":26319,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":70238,"ongoing_decode_tokens":70238,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2026,"cache_hit":0,"new_prefill":29903,"score_linear":70238.0,"score_lmetric":29903},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":29903,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-26624.0,"chosen_score_lmetric":0,"t_first_token":282413.642148609,"t_first_token_unix":1779713278.6560922,"t_done":282414.995434307,"t_done_unix":1779713280.0093775},{"request_id":"1363943:23:1488713:912","session_id":"1363943","input_length":72516,"t_proxy_recv":282419.051022202,"t_decision_unix":1779713284.0649636,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":70144,"affinity_cache_ratio":0.9672899773843014,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":70144,"estimated_new_tokens":2372,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1651,"cache_hit":8704,"new_prefill":63812,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":48452,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2478,"cache_hit":0,"new_prefill":72516,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":4096,"new_prefill":68420,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":2192,"cache_hit":0,"new_prefill":72516,"score_linear":131699.0,"score_lmetric":139191},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2289,"cache_hit":0,"new_prefill":72516,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2035,"cache_hit":70144,"new_prefill":2372,"score_linear":-70144.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":68420,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-70144.0,"chosen_score_lmetric":0,"t_first_token":282420.09597821,"t_first_token_unix":1779713285.1099217,"t_done":282422.076558112,"t_done_unix":1779713287.0905013},{"request_id":"1404395:7:1491186:914","session_id":"1404395","input_length":39569,"t_proxy_recv":282427.705629098,"t_decision_unix":1779713292.7195704,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":37888,"affinity_cache_ratio":0.9575172483509818,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":37888,"estimated_new_tokens":1681,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1651,"cache_hit":512,"new_prefill":39057,"score_linear":-512.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":39569,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2478,"cache_hit":37888,"new_prefill":1681,"score_linear":-37888.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":0,"new_prefill":39569,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":2192,"cache_hit":22528,"new_prefill":17041,"score_linear":109171.0,"score_lmetric":83716},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2289,"cache_hit":0,"new_prefill":39569,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":74939,"ongoing_decode_tokens":0,"pending_prefill_tokens":2235,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":39569,"score_linear":74939.0,"score_lmetric":41804},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":39569,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-37888.0,"chosen_score_lmetric":0,"t_first_token":282428.179480334,"t_first_token_unix":1779713293.193424,"t_done":282428.47234605,"t_done_unix":1779713293.4862897},{"request_id":"1363943:24:1491020:913","session_id":"1363943","input_length":74939,"t_proxy_recv":282427.311594006,"t_decision_unix":1779713292.3255353,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":72704,"affinity_cache_ratio":0.9701757429375892,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":72704,"estimated_new_tokens":2235,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1651,"cache_hit":8704,"new_prefill":66235,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":50875,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2478,"cache_hit":0,"new_prefill":74939,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":4096,"new_prefill":70843,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":2192,"cache_hit":0,"new_prefill":74939,"score_linear":131699.0,"score_lmetric":141614},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2289,"cache_hit":0,"new_prefill":74939,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":72704,"new_prefill":2235,"score_linear":-72704.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":70843,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-72704.0,"chosen_score_lmetric":0,"t_first_token":282428.532314255,"t_first_token_unix":1779713293.5462575,"t_done":282430.133157709,"t_done_unix":1779713295.1471012},{"request_id":"1363943:25:1493325:915","session_id":"1363943","input_length":75711,"t_proxy_recv":282434.793986851,"t_decision_unix":1779713299.8079278,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":74752,"affinity_cache_ratio":0.9873334125820554,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":74752,"estimated_new_tokens":959,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1651,"cache_hit":8704,"new_prefill":67007,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":51647,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2481,"cache_hit":0,"new_prefill":75711,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":4096,"new_prefill":71615,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":2192,"cache_hit":0,"new_prefill":75711,"score_linear":131699.0,"score_lmetric":142386},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2289,"cache_hit":0,"new_prefill":75711,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2044,"cache_hit":74752,"new_prefill":959,"score_linear":-74752.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":71615,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-74752.0,"chosen_score_lmetric":0,"t_first_token":282435.197455982,"t_first_token_unix":1779713300.2113998,"t_done":282437.392576413,"t_done_unix":1779713302.40652},{"request_id":"1313181:13:1349104:484","session_id":"1313181","input_length":131699,"t_proxy_recv":282398.687671479,"t_decision_unix":1779713263.7016127,"policy":"unified","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":65024,"affinity_cache_ratio":0.49373191899710706,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":65024,"estimated_new_tokens":66675,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1645,"cache_hit":0,"new_prefill":131699,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":131699,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38827,"ongoing_decode_tokens":38827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2476,"cache_hit":12800,"new_prefill":118899,"score_linear":26027.0,"score_lmetric":118899},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":0,"new_prefill":131699,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2192,"cache_hit":65024,"new_prefill":66675,"score_linear":-65024.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2289,"cache_hit":12800,"new_prefill":118899,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":65658,"ongoing_decode_tokens":0,"pending_prefill_tokens":4730,"num_requests":1,"active_p_offloads":0,"cached_blocks":2017,"cache_hit":0,"new_prefill":131699,"score_linear":65658.0,"score_lmetric":136429},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":62464,"new_prefill":69235,"score_linear":-62464.0,"score_lmetric":0}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":282439.162570518,"t_first_token_unix":1779713304.176514,"t_done":282440.851923758,"t_done_unix":1779713305.8658674},{"request_id":"1404395:8:1494219:916","session_id":"1404395","input_length":39820,"t_proxy_recv":282437.815884034,"t_decision_unix":1779713302.8298252,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":39424,"affinity_cache_ratio":0.9900552486187846,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":39424,"estimated_new_tokens":396,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1651,"cache_hit":512,"new_prefill":39308,"score_linear":-512.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":39820,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2481,"cache_hit":39424,"new_prefill":396,"score_linear":-39424.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":0,"new_prefill":39820,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":2192,"cache_hit":22528,"new_prefill":17292,"score_linear":109171.0,"score_lmetric":83967},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2289,"cache_hit":0,"new_prefill":39820,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2046,"cache_hit":0,"new_prefill":39820,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":39820,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-39424.0,"chosen_score_lmetric":0,"t_first_token":282437.963290579,"t_first_token_unix":1779713302.9772336,"t_done":282441.597510796,"t_done_unix":1779713306.611454},{"request_id":"1313181:13:1406095:783","session_id":"1313181","input_length":65279,"t_proxy_recv":282440.867770157,"t_decision_unix":1779713305.8817117,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9960936901606948,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":65024,"estimated_new_tokens":255,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1651,"cache_hit":0,"new_prefill":65279,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":65279,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":39820,"ongoing_decode_tokens":39820,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2481,"cache_hit":12800,"new_prefill":52479,"score_linear":27020.0,"score_lmetric":52479},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":0,"new_prefill":65279,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2196,"cache_hit":65024,"new_prefill":255,"score_linear":-65024.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2289,"cache_hit":12800,"new_prefill":52479,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2046,"cache_hit":0,"new_prefill":65279,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":62464,"new_prefill":2815,"score_linear":-62464.0,"score_lmetric":0}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":282440.965384893,"t_first_token_unix":1779713305.979328,"t_done":282443.152194313,"t_done_unix":1779713308.1661384},{"request_id":"1363943:26:1495648:917","session_id":"1363943","input_length":83861,"t_proxy_recv":282442.554695645,"t_decision_unix":1779713307.568637,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":75776,"affinity_cache_ratio":0.903590465174515,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75776,"estimated_new_tokens":8085,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1651,"cache_hit":8704,"new_prefill":75157,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":59797,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2482,"cache_hit":0,"new_prefill":83861,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":4096,"new_prefill":79765,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65279,"ongoing_decode_tokens":65279,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2196,"cache_hit":0,"new_prefill":83861,"score_linear":65279.0,"score_lmetric":83861},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2289,"cache_hit":0,"new_prefill":83861,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2046,"cache_hit":75776,"new_prefill":8085,"score_linear":-75776.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":79765,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-75776.0,"chosen_score_lmetric":0,"t_first_token":282446.579837735,"t_first_token_unix":1779713311.5937812,"t_done":282448.419557617,"t_done_unix":1779713313.4335012},{"request_id":"1363943:27:1497882:919","session_id":"1363943","input_length":84090,"t_proxy_recv":282449.829949539,"t_decision_unix":1779713314.8438854,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":83968,"affinity_cache_ratio":0.9985491735045784,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":83968,"estimated_new_tokens":122,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1651,"cache_hit":8704,"new_prefill":75386,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":60026,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":75760,"ongoing_decode_tokens":0,"pending_prefill_tokens":4080,"num_requests":1,"active_p_offloads":0,"cached_blocks":2482,"cache_hit":0,"new_prefill":84090,"score_linear":75760.0,"score_lmetric":88170},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":4096,"new_prefill":79994,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":2197,"cache_hit":0,"new_prefill":84090,"score_linear":139357.0,"score_lmetric":91863},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2289,"cache_hit":0,"new_prefill":84090,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2062,"cache_hit":83968,"new_prefill":122,"score_linear":-83968.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":79994,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-83968.0,"chosen_score_lmetric":0,"t_first_token":282449.971155341,"t_first_token_unix":1779713314.9850986,"t_done":282453.534749327,"t_done_unix":1779713318.548693},{"request_id":"1268861:14:1498281:920","session_id":"1268861","input_length":61910,"t_proxy_recv":282451.303864272,"t_decision_unix":1779713316.3178058,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":56832,"affinity_cache_ratio":0.9179777095784203,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":56832,"estimated_new_tokens":5078,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1651,"cache_hit":0,"new_prefill":61910,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":61910,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":75760,"ongoing_decode_tokens":0,"pending_prefill_tokens":4080,"num_requests":1,"active_p_offloads":0,"cached_blocks":2482,"cache_hit":0,"new_prefill":61910,"score_linear":75760.0,"score_lmetric":65990},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":0,"new_prefill":61910,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":2197,"cache_hit":21504,"new_prefill":40406,"score_linear":117853.0,"score_lmetric":48179},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2289,"cache_hit":56832,"new_prefill":5078,"score_linear":-56832.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":84090,"ongoing_decode_tokens":84090,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2062,"cache_hit":0,"new_prefill":61910,"score_linear":84090.0,"score_lmetric":61910},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":61910,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-56832.0,"chosen_score_lmetric":0,"t_first_token":282453.156573682,"t_first_token_unix":1779713318.1705172,"t_done":282453.728464449,"t_done_unix":1779713318.7424073},{"request_id":"1242838:7:1499476:921","session_id":"1242838","input_length":33352,"t_proxy_recv":282455.135384237,"t_decision_unix":1779713320.1493256,"policy":"unified","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":29696,"affinity_cache_ratio":0.8903813864236028,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":29696,"estimated_new_tokens":3656,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1651,"cache_hit":29696,"new_prefill":3656,"score_linear":-29696.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":33352,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":75760,"ongoing_decode_tokens":0,"pending_prefill_tokens":4080,"num_requests":1,"active_p_offloads":0,"cached_blocks":2482,"cache_hit":3584,"new_prefill":29768,"score_linear":72176.0,"score_lmetric":33848},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":0,"new_prefill":33352,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":2197,"cache_hit":0,"new_prefill":33352,"score_linear":139357.0,"score_lmetric":41125},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2299,"cache_hit":3584,"new_prefill":29768,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2062,"cache_hit":0,"new_prefill":33352,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":33352,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-29696.0,"chosen_score_lmetric":0,"t_first_token":282455.937847824,"t_first_token_unix":1779713320.9517913,"t_done":282457.956273596,"t_done_unix":1779713322.9702168},{"request_id":"1363943:28:1500984:922","session_id":"1363943","input_length":86984,"t_proxy_recv":282460.140010792,"t_decision_unix":1779713325.153952,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":83968,"affinity_cache_ratio":0.9653269566816886,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":83968,"estimated_new_tokens":3016,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":8704,"new_prefill":78280,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":62920,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":75760,"ongoing_decode_tokens":0,"pending_prefill_tokens":4080,"num_requests":1,"active_p_offloads":0,"cached_blocks":2482,"cache_hit":0,"new_prefill":86984,"score_linear":75760.0,"score_lmetric":91064},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":4096,"new_prefill":82888,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":2197,"cache_hit":0,"new_prefill":86984,"score_linear":139357.0,"score_lmetric":94757},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2299,"cache_hit":0,"new_prefill":86984,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2062,"cache_hit":83968,"new_prefill":3016,"score_linear":-83968.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":82888,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-83968.0,"chosen_score_lmetric":0,"t_first_token":282461.698420753,"t_first_token_unix":1779713326.7123642,"t_done":282463.880246663,"t_done_unix":1779713328.89419},{"request_id":"1388507:2:1496686:918","session_id":"1388507","input_length":75760,"t_proxy_recv":282445.920749666,"t_decision_unix":1779713310.9346912,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":71680,"affinity_cache_ratio":0.9461457233368532,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71680,"estimated_new_tokens":4080,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1651,"cache_hit":0,"new_prefill":75760,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":75760,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2482,"cache_hit":71680,"new_prefill":4080,"score_linear":-71680.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":0,"new_prefill":75760,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":2197,"cache_hit":0,"new_prefill":75760,"score_linear":139357.0,"score_lmetric":83533},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2289,"cache_hit":0,"new_prefill":75760,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":83861,"ongoing_decode_tokens":0,"pending_prefill_tokens":8085,"num_requests":1,"active_p_offloads":0,"cached_blocks":2046,"cache_hit":0,"new_prefill":75760,"score_linear":83861.0,"score_lmetric":83845},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":75760,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-71680.0,"chosen_score_lmetric":0,"t_first_token":282465.588289464,"t_first_token_unix":1779713330.6022332,"t_done":282467.216968832,"t_done_unix":1779713332.230912},{"request_id":"1342921:8:1503584:925","session_id":"1342921","input_length":78722,"t_proxy_recv":282468.998427881,"t_decision_unix":1779713334.0123692,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":76288,"affinity_cache_ratio":0.9690810700947639,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":76288,"estimated_new_tokens":2434,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":78722,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":78722,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2490,"cache_hit":0,"new_prefill":78722,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":76288,"new_prefill":2434,"score_linear":-76288.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":2197,"cache_hit":71168,"new_prefill":7554,"score_linear":68189.0,"score_lmetric":15327},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103893,"ongoing_decode_tokens":0,"pending_prefill_tokens":10709,"num_requests":1,"active_p_offloads":0,"cached_blocks":2299,"cache_hit":0,"new_prefill":78722,"score_linear":103893.0,"score_lmetric":89431},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89469,"ongoing_decode_tokens":0,"pending_prefill_tokens":2429,"num_requests":1,"active_p_offloads":0,"cached_blocks":2068,"cache_hit":0,"new_prefill":78722,"score_linear":89469.0,"score_lmetric":81151},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":78722,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-76288.0,"chosen_score_lmetric":0,"t_first_token":282470.135855093,"t_first_token_unix":1779713335.1497984,"t_done":282470.310054779,"t_done_unix":1779713335.3239985},{"request_id":"1363943:29:1503515:924","session_id":"1363943","input_length":89469,"t_proxy_recv":282468.741935259,"t_decision_unix":1779713333.7558763,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":87040,"affinity_cache_ratio":0.972850931607596,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":87040,"estimated_new_tokens":2429,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":8704,"new_prefill":80765,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":65405,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2490,"cache_hit":0,"new_prefill":89469,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":4096,"new_prefill":85373,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":2197,"cache_hit":0,"new_prefill":89469,"score_linear":139357.0,"score_lmetric":97242},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103893,"ongoing_decode_tokens":0,"pending_prefill_tokens":10709,"num_requests":1,"active_p_offloads":0,"cached_blocks":2299,"cache_hit":0,"new_prefill":89469,"score_linear":103893.0,"score_lmetric":100178},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2068,"cache_hit":87040,"new_prefill":2429,"score_linear":-87040.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":85373,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-87040.0,"chosen_score_lmetric":0,"t_first_token":282470.179843319,"t_first_token_unix":1779713335.1937876,"t_done":282472.520929006,"t_done_unix":1779713337.5348723},{"request_id":"1399948:14:1504493:926","session_id":"1399948","input_length":14189,"t_proxy_recv":282471.773193771,"t_decision_unix":1779713336.7871351,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":11776,"affinity_cache_ratio":0.829938684896751,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":2413,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":14189,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":14189,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2490,"cache_hit":0,"new_prefill":14189,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2113,"cache_hit":0,"new_prefill":14189,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":2197,"cache_hit":0,"new_prefill":14189,"score_linear":139357.0,"score_lmetric":21962},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103893,"ongoing_decode_tokens":0,"pending_prefill_tokens":10709,"num_requests":1,"active_p_offloads":0,"cached_blocks":2299,"cache_hit":0,"new_prefill":14189,"score_linear":103893.0,"score_lmetric":24898},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89469,"ongoing_decode_tokens":89469,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2068,"cache_hit":11776,"new_prefill":2413,"score_linear":77693.0,"score_lmetric":2413},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":14189,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":77693.0,"chosen_score_lmetric":2413,"t_first_token":282472.07054371,"t_first_token_unix":1779713337.0844872,"t_done":282472.579972702,"t_done_unix":1779713337.593916},{"request_id":"1363943:30:1506171:927","session_id":"1363943","input_length":89713,"t_proxy_recv":282477.579868796,"t_decision_unix":1779713342.5938098,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":89600,"affinity_cache_ratio":0.9987404278086788,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":89600,"estimated_new_tokens":113,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":8704,"new_prefill":81009,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":65649,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2490,"cache_hit":0,"new_prefill":89713,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2113,"cache_hit":4096,"new_prefill":85617,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":2197,"cache_hit":0,"new_prefill":89713,"score_linear":139357.0,"score_lmetric":97486},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103893,"ongoing_decode_tokens":0,"pending_prefill_tokens":10709,"num_requests":1,"active_p_offloads":0,"cached_blocks":2299,"cache_hit":0,"new_prefill":89713,"score_linear":103893.0,"score_lmetric":100422},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2077,"cache_hit":89600,"new_prefill":113,"score_linear":-89600.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":85617,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-89600.0,"chosen_score_lmetric":0,"t_first_token":282477.723129246,"t_first_token_unix":1779713342.7370727,"t_done":282478.383624761,"t_done_unix":1779713343.3975682},{"request_id":"1313181:14:1351342:491","session_id":"1313181","input_length":139357,"t_proxy_recv":282443.177212052,"t_decision_unix":1779713308.1911538,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":131584,"affinity_cache_ratio":0.9442223928471479,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":131584,"estimated_new_tokens":7773,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1651,"cache_hit":0,"new_prefill":139357,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":139357,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2482,"cache_hit":12800,"new_prefill":126557,"score_linear":-12800.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":0,"new_prefill":139357,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2197,"cache_hit":131584,"new_prefill":7773,"score_linear":-131584.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2289,"cache_hit":12800,"new_prefill":126557,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":83861,"ongoing_decode_tokens":0,"pending_prefill_tokens":8085,"num_requests":1,"active_p_offloads":0,"cached_blocks":2046,"cache_hit":0,"new_prefill":139357,"score_linear":83861.0,"score_lmetric":147442},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":62464,"new_prefill":76893,"score_linear":-62464.0,"score_lmetric":0}],"chosen_score_linear":-131584.0,"chosen_score_lmetric":0,"t_first_token":282489.63227411,"t_first_token_unix":1779713354.646218,"t_done":282498.029455244,"t_done_unix":1779713363.0433996},{"request_id":"1399948:15:1512300:930","session_id":"1399948","input_length":15669,"t_proxy_recv":282498.095274368,"t_decision_unix":1779713363.1092155,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":13824,"affinity_cache_ratio":0.8822515795519816,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":13824,"estimated_new_tokens":1845,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":15669,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":15669,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2490,"cache_hit":0,"new_prefill":15669,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2113,"cache_hit":0,"new_prefill":15669,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65876,"ongoing_decode_tokens":0,"pending_prefill_tokens":340,"num_requests":1,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":15669,"score_linear":65876.0,"score_lmetric":16009},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166928,"ongoing_decode_tokens":0,"pending_prefill_tokens":11792,"num_requests":2,"active_p_offloads":0,"cached_blocks":2299,"cache_hit":0,"new_prefill":15669,"score_linear":166928.0,"score_lmetric":54922},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2077,"cache_hit":13824,"new_prefill":1845,"score_linear":75966.0,"score_lmetric":1845},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":15669,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":75966.0,"chosen_score_lmetric":1845,"t_first_token":282498.363188202,"t_first_token_unix":1779713363.3771327,"t_done":282498.861242559,"t_done_unix":1779713363.875186},{"request_id":"1313181:14:1408792:803","session_id":"1313181","input_length":65876,"t_proxy_recv":282498.040066763,"t_decision_unix":1779713363.0540078,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":65536,"affinity_cache_ratio":0.9948387880259882,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":65536,"estimated_new_tokens":340,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":65876,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":65876,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2490,"cache_hit":12800,"new_prefill":53076,"score_linear":-12800.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2113,"cache_hit":0,"new_prefill":65876,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":65536,"new_prefill":340,"score_linear":-65536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166928,"ongoing_decode_tokens":0,"pending_prefill_tokens":11792,"num_requests":2,"active_p_offloads":0,"cached_blocks":2299,"cache_hit":12800,"new_prefill":53076,"score_linear":154128.0,"score_lmetric":129736},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2077,"cache_hit":0,"new_prefill":65876,"score_linear":89790.0,"score_lmetric":65876},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":62464,"new_prefill":3412,"score_linear":-62464.0,"score_lmetric":0}],"chosen_score_linear":-65536.0,"chosen_score_lmetric":0,"t_first_token":282498.147225959,"t_first_token_unix":1779713363.16117,"t_done":282501.128439441,"t_done_unix":1779713366.1423852},{"request_id":"1268861:15:1508874:928","session_id":"1268861","input_length":63035,"t_proxy_recv":282486.628136761,"t_decision_unix":1779713351.6420782,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":61952,"affinity_cache_ratio":0.9828190687713175,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":61952,"estimated_new_tokens":1083,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":63035,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":63035,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2490,"cache_hit":0,"new_prefill":63035,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2113,"cache_hit":0,"new_prefill":63035,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":2197,"cache_hit":21504,"new_prefill":41531,"score_linear":117853.0,"score_lmetric":49304},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103893,"ongoing_decode_tokens":0,"pending_prefill_tokens":10709,"num_requests":1,"active_p_offloads":0,"cached_blocks":2299,"cache_hit":61952,"new_prefill":1083,"score_linear":41941.0,"score_lmetric":11792},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2077,"cache_hit":0,"new_prefill":63035,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":63035,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":41941.0,"chosen_score_lmetric":11792,"t_first_token":282499.251353706,"t_first_token_unix":1779713364.2652974,"t_done":282501.790874883,"t_done_unix":1779713366.8048186},{"request_id":"1277428:9:1513231:932","session_id":"1277428","input_length":44364,"t_proxy_recv":282501.106706563,"t_decision_unix":1779713366.1206474,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":42496,"affinity_cache_ratio":0.9578937877558381,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":42496,"estimated_new_tokens":1868,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":44364,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":4608,"new_prefill":39756,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":130532,"ongoing_decode_tokens":0,"pending_prefill_tokens":996,"num_requests":1,"active_p_offloads":0,"cached_blocks":2490,"cache_hit":0,"new_prefill":44364,"score_linear":130532.0,"score_lmetric":45360},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2113,"cache_hit":42496,"new_prefill":1868,"score_linear":-42496.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65876,"ongoing_decode_tokens":65876,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":44364,"score_linear":65876.0,"score_lmetric":44364},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166928,"ongoing_decode_tokens":166928,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2299,"cache_hit":0,"new_prefill":44364,"score_linear":166928.0,"score_lmetric":88728},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2080,"cache_hit":0,"new_prefill":44364,"score_linear":89790.0,"score_lmetric":44364},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":44364,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-42496.0,"chosen_score_lmetric":0,"t_first_token":282501.628292874,"t_first_token_unix":1779713366.6422367,"t_done":282501.928562767,"t_done_unix":1779713366.942506},{"request_id":"1397515:2:1502033:923","session_id":"1397515","input_length":103893,"t_proxy_recv":282463.702575127,"t_decision_unix":1779713328.7165163,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":93184,"affinity_cache_ratio":0.8969227955685176,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":93184,"estimated_new_tokens":10709,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":103893,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":512,"new_prefill":103381,"score_linear":-512.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":75760,"ongoing_decode_tokens":0,"pending_prefill_tokens":4080,"num_requests":1,"active_p_offloads":0,"cached_blocks":2482,"cache_hit":0,"new_prefill":103893,"score_linear":75760.0,"score_lmetric":107973},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2109,"cache_hit":0,"new_prefill":103893,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":2197,"cache_hit":0,"new_prefill":103893,"score_linear":139357.0,"score_lmetric":111666},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2299,"cache_hit":93184,"new_prefill":10709,"score_linear":-93184.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":86984,"ongoing_decode_tokens":86984,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2062,"cache_hit":0,"new_prefill":103893,"score_linear":86984.0,"score_lmetric":103893},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":103893,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-93184.0,"chosen_score_lmetric":0,"t_first_token":282499.248055048,"t_first_token_unix":1779713364.261999,"t_done":282503.755950322,"t_done_unix":1779713368.769894},{"request_id":"1399948:16:1513839:933","session_id":"1399948","input_length":15913,"t_proxy_recv":282503.270022192,"t_decision_unix":1779713368.2839634,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":15360,"affinity_cache_ratio":0.9652485389304343,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":15360,"estimated_new_tokens":553,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":15913,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":15913,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":130532,"ongoing_decode_tokens":0,"pending_prefill_tokens":996,"num_requests":1,"active_p_offloads":0,"cached_blocks":2490,"cache_hit":0,"new_prefill":15913,"score_linear":130532.0,"score_lmetric":16909},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2116,"cache_hit":0,"new_prefill":15913,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":66259,"ongoing_decode_tokens":66259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2214,"cache_hit":0,"new_prefill":15913,"score_linear":66259.0,"score_lmetric":15913},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103893,"ongoing_decode_tokens":103893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2301,"cache_hit":0,"new_prefill":15913,"score_linear":103893.0,"score_lmetric":15913},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2080,"cache_hit":15360,"new_prefill":553,"score_linear":74430.0,"score_lmetric":553},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":15913,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":74430.0,"chosen_score_lmetric":553,"t_first_token":282503.355329224,"t_first_token_unix":1779713368.3692732,"t_done":282504.22785892,"t_done_unix":1779713369.2418025},{"request_id":"1313181:15:1411723:818","session_id":"1313181","input_length":66259,"t_proxy_recv":282501.140164102,"t_decision_unix":1779713366.1541054,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":66048,"affinity_cache_ratio":0.9968155269472826,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":66048,"estimated_new_tokens":211,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":66259,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":66259,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":130532,"ongoing_decode_tokens":0,"pending_prefill_tokens":996,"num_requests":1,"active_p_offloads":0,"cached_blocks":2490,"cache_hit":12800,"new_prefill":53459,"score_linear":117732.0,"score_lmetric":54455},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":44364,"ongoing_decode_tokens":0,"pending_prefill_tokens":1868,"num_requests":1,"active_p_offloads":0,"cached_blocks":2113,"cache_hit":0,"new_prefill":66259,"score_linear":44364.0,"score_lmetric":68127},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2214,"cache_hit":66048,"new_prefill":211,"score_linear":-66048.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166928,"ongoing_decode_tokens":166928,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2299,"cache_hit":12800,"new_prefill":53459,"score_linear":154128.0,"score_lmetric":106918},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2080,"cache_hit":0,"new_prefill":66259,"score_linear":89790.0,"score_lmetric":66259},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":62464,"new_prefill":3795,"score_linear":-62464.0,"score_lmetric":0}],"chosen_score_linear":-66048.0,"chosen_score_lmetric":0,"t_first_token":282501.266551274,"t_first_token_unix":1779713366.280496,"t_done":282504.528957799,"t_done_unix":1779713369.5429013},{"request_id":"1277428:10:1517408:935","session_id":"1277428","input_length":45851,"t_proxy_recv":282514.903174422,"t_decision_unix":1779713379.9171154,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":44032,"affinity_cache_ratio":0.9603280190181239,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":44032,"estimated_new_tokens":1819,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":45851,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":4608,"new_prefill":41243,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":130532,"ongoing_decode_tokens":0,"pending_prefill_tokens":996,"num_requests":1,"active_p_offloads":0,"cached_blocks":2490,"cache_hit":0,"new_prefill":45851,"score_linear":130532.0,"score_lmetric":46847},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2116,"cache_hit":44032,"new_prefill":1819,"score_linear":-44032.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":45851,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2322,"cache_hit":0,"new_prefill":45851,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2081,"cache_hit":0,"new_prefill":45851,"score_linear":89790.0,"score_lmetric":45851},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":45851,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-44032.0,"chosen_score_lmetric":0,"t_first_token":282515.435524843,"t_first_token_unix":1779713380.4494689,"t_done":282517.536906898,"t_done_unix":1779713382.5508504},{"request_id":"1399948:17:1518102:936","session_id":"1399948","input_length":16604,"t_proxy_recv":282517.383678672,"t_decision_unix":1779713382.39762,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":15872,"affinity_cache_ratio":0.9559142375331245,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":15872,"estimated_new_tokens":732,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":16604,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":16604,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":130532,"ongoing_decode_tokens":0,"pending_prefill_tokens":996,"num_requests":1,"active_p_offloads":0,"cached_blocks":2490,"cache_hit":0,"new_prefill":16604,"score_linear":130532.0,"score_lmetric":17600},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":45851,"ongoing_decode_tokens":45851,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2116,"cache_hit":0,"new_prefill":16604,"score_linear":45851.0,"score_lmetric":16604},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":16604,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2322,"cache_hit":0,"new_prefill":16604,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2081,"cache_hit":15872,"new_prefill":732,"score_linear":73918.0,"score_lmetric":732},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":16604,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":73918.0,"chosen_score_lmetric":732,"t_first_token":282517.61904822,"t_first_token_unix":1779713382.6329918,"t_done":282518.347052222,"t_done_unix":1779713383.3609955},{"request_id":"1356885:4:1512902:931","session_id":"1356885","input_length":130532,"t_proxy_recv":282499.982525481,"t_decision_unix":1779713364.9964662,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":129536,"affinity_cache_ratio":0.9923696871265284,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":129536,"estimated_new_tokens":996,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":130532,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":130532,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2490,"cache_hit":129536,"new_prefill":996,"score_linear":-129536.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2113,"cache_hit":6656,"new_prefill":123876,"score_linear":-6656.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65876,"ongoing_decode_tokens":65876,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":130532,"score_linear":65876.0,"score_lmetric":130532},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166928,"ongoing_decode_tokens":166928,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2299,"cache_hit":0,"new_prefill":130532,"score_linear":166928.0,"score_lmetric":261064},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2080,"cache_hit":0,"new_prefill":130532,"score_linear":89790.0,"score_lmetric":130532},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":130532,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-129536.0,"chosen_score_lmetric":0,"t_first_token":282521.769792263,"t_first_token_unix":1779713386.783736,"t_done":282525.20390448,"t_done_unix":1779713390.217848},{"request_id":"1277428:11:1520449:937","session_id":"1277428","input_length":46361,"t_proxy_recv":282524.887082139,"t_decision_unix":1779713389.9010234,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":46080,"affinity_cache_ratio":0.9939388710338432,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":46080,"estimated_new_tokens":281,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":46361,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":4608,"new_prefill":41753,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":130532,"ongoing_decode_tokens":130532,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2490,"cache_hit":0,"new_prefill":46361,"score_linear":130532.0,"score_lmetric":46361},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2120,"cache_hit":46080,"new_prefill":281,"score_linear":-46080.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":46361,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2322,"cache_hit":0,"new_prefill":46361,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2082,"cache_hit":0,"new_prefill":46361,"score_linear":89790.0,"score_lmetric":46361},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":46361,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-46080.0,"chosen_score_lmetric":0,"t_first_token":282525.054204611,"t_first_token_unix":1779713390.0681484,"t_done":282525.450421707,"t_done_unix":1779713390.464365},{"request_id":"1363943:31:1509815:929","session_id":"1363943","input_length":89790,"t_proxy_recv":282489.656680985,"t_decision_unix":1779713354.6706223,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":89600,"affinity_cache_ratio":0.9978839514422542,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":89600,"estimated_new_tokens":190,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":8704,"new_prefill":81086,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":24064,"new_prefill":65726,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2490,"cache_hit":0,"new_prefill":89790,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2113,"cache_hit":4096,"new_prefill":85694,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":139357,"ongoing_decode_tokens":139357,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2197,"cache_hit":0,"new_prefill":89790,"score_linear":139357.0,"score_lmetric":89790},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166928,"ongoing_decode_tokens":0,"pending_prefill_tokens":11792,"num_requests":2,"active_p_offloads":0,"cached_blocks":2299,"cache_hit":0,"new_prefill":89790,"score_linear":166928.0,"score_lmetric":203164},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2077,"cache_hit":89600,"new_prefill":190,"score_linear":-89600.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":85694,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-89600.0,"chosen_score_lmetric":0,"t_first_token":282489.784859637,"t_first_token_unix":1779713354.7988036,"t_done":282525.921654011,"t_done_unix":1779713390.935598},{"request_id":"1399948:18:1524432:938","session_id":"1399948","input_length":17276,"t_proxy_recv":282538.443296875,"t_decision_unix":1779713403.4572377,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":16384,"affinity_cache_ratio":0.948367677703172,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":16384,"estimated_new_tokens":892,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2120,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2322,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":16384,"new_prefill":892,"score_linear":-16384.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-16384.0,"chosen_score_lmetric":0,"t_first_token":282538.577179275,"t_first_token_unix":1779713403.591123,"t_done":282538.930901294,"t_done_unix":1779713403.944845},{"request_id":"1342921:9:1525898:939","session_id":"1342921","input_length":79086,"t_proxy_recv":282543.478964291,"t_decision_unix":1779713408.4929051,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":78336,"affinity_cache_ratio":0.9905166527577574,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":78336,"estimated_new_tokens":750,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2120,"cache_hit":78336,"new_prefill":750,"score_linear":-78336.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":71168,"new_prefill":7918,"score_linear":-71168.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2322,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2084,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-78336.0,"chosen_score_lmetric":0,"t_first_token":282543.878511132,"t_first_token_unix":1779713408.892455,"t_done":282543.959564647,"t_done_unix":1779713408.9735084},{"request_id":"1399948:19:1526739:941","session_id":"1399948","input_length":18091,"t_proxy_recv":282546.356775935,"t_decision_unix":1779713411.3707168,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":16896,"affinity_cache_ratio":0.9339450555524846,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":16896,"estimated_new_tokens":1195,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":18091,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":18091,"score_linear":87576.0,"score_lmetric":28867},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":18091,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":18091,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":18091,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2322,"cache_hit":0,"new_prefill":18091,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2084,"cache_hit":16896,"new_prefill":1195,"score_linear":-16896.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":18091,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-16896.0,"chosen_score_lmetric":0,"t_first_token":282546.529245987,"t_first_token_unix":1779713411.5431898,"t_done":282546.811455169,"t_done_unix":1779713411.825399},{"request_id":"1399948:20:1527857:942","session_id":"1399948","input_length":18164,"t_proxy_recv":282549.855768818,"t_decision_unix":1779713414.8697097,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":17920,"affinity_cache_ratio":0.9865668354987888,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":17920,"estimated_new_tokens":244,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":18164,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":18164,"score_linear":87576.0,"score_lmetric":28940},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":18164,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":18164,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":18164,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2322,"cache_hit":0,"new_prefill":18164,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2086,"cache_hit":17920,"new_prefill":244,"score_linear":-17920.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":18164,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-17920.0,"chosen_score_lmetric":0,"t_first_token":282549.903307788,"t_first_token_unix":1779713414.917251,"t_done":282550.438333414,"t_done_unix":1779713415.4522767},{"request_id":"1399948:21:1528978:944","session_id":"1399948","input_length":18388,"t_proxy_recv":282553.803534029,"t_decision_unix":1779713418.8174756,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":17920,"affinity_cache_ratio":0.9745486186643463,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":17920,"estimated_new_tokens":468,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":18388,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":18388,"score_linear":87576.0,"score_lmetric":29164},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":18388,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":18388,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":18388,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2322,"cache_hit":0,"new_prefill":18388,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":36568,"ongoing_decode_tokens":0,"pending_prefill_tokens":36568,"num_requests":1,"active_p_offloads":0,"cached_blocks":2086,"cache_hit":17920,"new_prefill":468,"score_linear":18648.0,"score_lmetric":37036},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":18388,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":18648.0,"chosen_score_lmetric":37036,"t_first_token":282556.570304252,"t_first_token_unix":1779713421.5842474,"t_done":282557.391332746,"t_done_unix":1779713422.4052763},{"request_id":"1399948:22:1530122:947","session_id":"1399948","input_length":18563,"t_proxy_recv":282557.629214519,"t_decision_unix":1779713422.6431556,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":18432,"affinity_cache_ratio":0.992942951031622,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":18432,"estimated_new_tokens":131,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":18563,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":18563,"score_linear":87576.0,"score_lmetric":29339},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":18563,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":18563,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":18563,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":71192,"ongoing_decode_tokens":0,"pending_prefill_tokens":8216,"num_requests":1,"active_p_offloads":0,"cached_blocks":2322,"cache_hit":0,"new_prefill":18563,"score_linear":71192.0,"score_lmetric":26779},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":36568,"ongoing_decode_tokens":36568,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2087,"cache_hit":18432,"new_prefill":131,"score_linear":18136.0,"score_lmetric":131},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":18563,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":18136.0,"chosen_score_lmetric":131,"t_first_token":282557.695459541,"t_first_token_unix":1779713422.709403,"t_done":282558.061792449,"t_done_unix":1779713423.0757358},{"request_id":"1243831:2:1528168:943","session_id":"1243831","input_length":36568,"t_proxy_recv":282551.06813191,"t_decision_unix":1779713416.082073,"policy":"unified","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":6,"affinity_cache_hit":11264,"affinity_cache_ratio":0.30802887770728504,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":36568,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":11264,"new_prefill":25304,"score_linear":76312.0,"score_lmetric":36080},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2322,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2086,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":282556.566486771,"t_first_token_unix":1779713421.5804312,"t_done":282561.076588412,"t_done_unix":1779713426.0905316},{"request_id":"1268861:16:1529894:946","session_id":"1268861","input_length":71192,"t_proxy_recv":282556.815581198,"t_decision_unix":1779713421.8295221,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":62976,"affinity_cache_ratio":0.8845937745814136,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":62976,"estimated_new_tokens":8216,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":71192,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":71192,"score_linear":87576.0,"score_lmetric":81968},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":71192,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":71192,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":21504,"new_prefill":49688,"score_linear":-21504.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2322,"cache_hit":62976,"new_prefill":8216,"score_linear":-62976.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":54956,"ongoing_decode_tokens":54956,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2086,"cache_hit":0,"new_prefill":71192,"score_linear":54956.0,"score_lmetric":142384},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":71192,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-62976.0,"chosen_score_lmetric":0,"t_first_token":282560.272401095,"t_first_token_unix":1779713425.2863448,"t_done":282561.408830432,"t_done_unix":1779713426.4227743},{"request_id":"1243831:3:1270336:110","session_id":"1243831","input_length":11584,"t_proxy_recv":282561.080181804,"t_decision_unix":1779713426.0941234,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":11264,"affinity_cache_ratio":0.9723756906077348,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":320,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":11584,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":11264,"new_prefill":320,"score_linear":76312.0,"score_lmetric":11096},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":11584,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":11584,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":11584,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":71192,"ongoing_decode_tokens":71192,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2322,"cache_hit":0,"new_prefill":11584,"score_linear":71192.0,"score_lmetric":11584},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2159,"cache_hit":11264,"new_prefill":320,"score_linear":-11264.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":11584,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11264.0,"chosen_score_lmetric":0,"t_first_token":282561.111215616,"t_first_token_unix":1779713426.1251595,"t_done":282561.948124908,"t_done_unix":1779713426.9620683},{"request_id":"1243831:4:1284306:183","session_id":"1243831","input_length":14515,"t_proxy_recv":282561.953598307,"t_decision_unix":1779713426.9675395,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":11264,"affinity_cache_ratio":0.7760248019290389,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":3251,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":14515,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":11264,"new_prefill":3251,"score_linear":76312.0,"score_lmetric":14027},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":14515,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":14515,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":14515,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":14515,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2159,"cache_hit":11264,"new_prefill":3251,"score_linear":-11264.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":14515,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11264.0,"chosen_score_lmetric":0,"t_first_token":282562.322625159,"t_first_token_unix":1779713427.3365712,"t_done":282562.749990541,"t_done_unix":1779713427.763934},{"request_id":"1243831:5:1303288:273","session_id":"1243831","input_length":14627,"t_proxy_recv":282562.754215862,"t_decision_unix":1779713427.768157,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":14336,"affinity_cache_ratio":0.980105284747385,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":291,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":14627,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":11264,"new_prefill":3363,"score_linear":76312.0,"score_lmetric":14139},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":14627,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":14627,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":14627,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":14627,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2165,"cache_hit":14336,"new_prefill":291,"score_linear":-14336.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":14627,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-14336.0,"chosen_score_lmetric":0,"t_first_token":282562.803516586,"t_first_token_unix":1779713427.8174603,"t_done":282563.253995171,"t_done_unix":1779713428.2679389},{"request_id":"1243831:6:1313694:320","session_id":"1243831","input_length":17410,"t_proxy_recv":282563.259937952,"t_decision_unix":1779713428.2738793,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":14336,"affinity_cache_ratio":0.8234348075818495,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":3074,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":17410,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":11264,"new_prefill":6146,"score_linear":76312.0,"score_lmetric":16922},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":17410,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":17410,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":17410,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":17410,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2165,"cache_hit":14336,"new_prefill":3074,"score_linear":-14336.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":17410,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-14336.0,"chosen_score_lmetric":0,"t_first_token":282563.67128847,"t_first_token_unix":1779713428.685232,"t_done":282564.126340903,"t_done_unix":1779713429.1402843},{"request_id":"1243831:7:1323865:366","session_id":"1243831","input_length":19460,"t_proxy_recv":282564.131929669,"t_decision_unix":1779713429.1458712,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":17408,"affinity_cache_ratio":0.8945529290853032,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":17408,"estimated_new_tokens":2052,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":19460,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":11264,"new_prefill":8196,"score_linear":76312.0,"score_lmetric":18972},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":19460,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":19460,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":19460,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":19460,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":17408,"new_prefill":2052,"score_linear":-17408.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":19460,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-17408.0,"chosen_score_lmetric":0,"t_first_token":282564.484307684,"t_first_token_unix":1779713429.498251,"t_done":282565.086041565,"t_done_unix":1779713430.0999851},{"request_id":"1243831:8:1333049:414","session_id":"1243831","input_length":24512,"t_proxy_recv":282565.098710894,"t_decision_unix":1779713430.112652,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":19456,"affinity_cache_ratio":0.793733681462141,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":19456,"estimated_new_tokens":5056,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":24512,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":11264,"new_prefill":13248,"score_linear":76312.0,"score_lmetric":24024},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":24512,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":24512,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":24512,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":24512,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2175,"cache_hit":19456,"new_prefill":5056,"score_linear":-19456.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":24512,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-19456.0,"chosen_score_lmetric":0,"t_first_token":282565.970270694,"t_first_token_unix":1779713430.9842145,"t_done":282566.645821104,"t_done_unix":1779713431.6597648},{"request_id":"1243831:9:1348240:478","session_id":"1243831","input_length":25075,"t_proxy_recv":282566.651742576,"t_decision_unix":1779713431.6656837,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":24576,"affinity_cache_ratio":0.980099700897308,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24576,"estimated_new_tokens":499,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":25075,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":11264,"new_prefill":13811,"score_linear":76312.0,"score_lmetric":24587},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":25075,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":25075,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":25075,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":25075,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2185,"cache_hit":24576,"new_prefill":499,"score_linear":-24576.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":25075,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-24576.0,"chosen_score_lmetric":0,"t_first_token":282566.813981068,"t_first_token_unix":1779713431.8279247,"t_done":282567.220907391,"t_done_unix":1779713432.2348511},{"request_id":"1243831:10:1358921:521","session_id":"1243831","input_length":26343,"t_proxy_recv":282567.226774734,"t_decision_unix":1779713432.240716,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":25088,"affinity_cache_ratio":0.9523592605246175,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":25088,"estimated_new_tokens":1255,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":26343,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":11264,"new_prefill":15079,"score_linear":76312.0,"score_lmetric":25855},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":26343,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":26343,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":26343,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":26343,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2186,"cache_hit":25088,"new_prefill":1255,"score_linear":-25088.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":26343,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-25088.0,"chosen_score_lmetric":0,"t_first_token":282567.513961911,"t_first_token_unix":1779713432.5279057,"t_done":282568.206755382,"t_done_unix":1779713433.2206988},{"request_id":"1243831:11:1371191:594","session_id":"1243831","input_length":26818,"t_proxy_recv":282568.212714833,"t_decision_unix":1779713433.2266564,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":26112,"affinity_cache_ratio":0.9736743977925274,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":26112,"estimated_new_tokens":706,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":26818,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":11264,"new_prefill":15554,"score_linear":76312.0,"score_lmetric":26330},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":26818,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":26818,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":26818,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":26818,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2188,"cache_hit":26112,"new_prefill":706,"score_linear":-26112.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":26818,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-26112.0,"chosen_score_lmetric":0,"t_first_token":282568.3756892,"t_first_token_unix":1779713433.3896325,"t_done":282568.833326655,"t_done_unix":1779713433.8472703},{"request_id":"1243831:12:1385174:681","session_id":"1243831","input_length":27164,"t_proxy_recv":282568.839385891,"t_decision_unix":1779713433.8533273,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":26624,"affinity_cache_ratio":0.9801207480488883,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":26624,"estimated_new_tokens":540,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":27164,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":11264,"new_prefill":15900,"score_linear":76312.0,"score_lmetric":26676},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":27164,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":27164,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":27164,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":27164,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":26624,"new_prefill":540,"score_linear":-26624.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":27164,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-26624.0,"chosen_score_lmetric":0,"t_first_token":282568.956372992,"t_first_token_unix":1779713433.9703166,"t_done":282569.360277254,"t_done_unix":1779713434.3742206},{"request_id":"1243831:13:1398072:738","session_id":"1243831","input_length":27286,"t_proxy_recv":282569.366230756,"t_decision_unix":1779713434.380172,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9945026753646559,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":27136,"estimated_new_tokens":150,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":27286,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":11264,"new_prefill":16022,"score_linear":76312.0,"score_lmetric":26798},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":27286,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":27286,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":27286,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":27286,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2190,"cache_hit":27136,"new_prefill":150,"score_linear":-27136.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":27286,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-27136.0,"chosen_score_lmetric":0,"t_first_token":282569.440636625,"t_first_token_unix":1779713434.4545803,"t_done":282569.972355524,"t_done_unix":1779713434.9862983},{"request_id":"1243831:14:1411189:814","session_id":"1243831","input_length":28085,"t_proxy_recv":282569.978264448,"t_decision_unix":1779713434.9922059,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9662097204913654,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":27136,"estimated_new_tokens":949,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":28085,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":159563,"ongoing_decode_tokens":0,"pending_prefill_tokens":18763,"num_requests":2,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":11264,"new_prefill":16821,"score_linear":148299.0,"score_lmetric":71168},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":130906,"ongoing_decode_tokens":130906,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":28085,"score_linear":130906.0,"score_lmetric":28085},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":28085,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":28085,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":28085,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2190,"cache_hit":27136,"new_prefill":949,"score_linear":-27136.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":28085,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-27136.0,"chosen_score_lmetric":0,"t_first_token":282570.208868302,"t_first_token_unix":1779713435.2228122,"t_done":282570.740965521,"t_done_unix":1779713435.7549093},{"request_id":"1243831:15:1427593:849","session_id":"1243831","input_length":29382,"t_proxy_recv":282570.747665795,"t_decision_unix":1779713435.761607,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":28160,"affinity_cache_ratio":0.9584099108297597,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":28160,"estimated_new_tokens":1222,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":29382,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":159563,"ongoing_decode_tokens":0,"pending_prefill_tokens":18763,"num_requests":2,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":11264,"new_prefill":18118,"score_linear":148299.0,"score_lmetric":73762},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":130906,"ongoing_decode_tokens":130906,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":29382,"score_linear":130906.0,"score_lmetric":29382},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":29382,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":29382,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":29382,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2192,"cache_hit":28160,"new_prefill":1222,"score_linear":-28160.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":29382,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-28160.0,"chosen_score_lmetric":0,"t_first_token":282571.056846454,"t_first_token_unix":1779713436.0707908,"t_done":282571.911516874,"t_done_unix":1779713436.9254603},{"request_id":"1356885:5:1533696:948","session_id":"1356885","input_length":130906,"t_proxy_recv":282569.469055941,"t_decision_unix":1779713434.482997,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":130560,"affinity_cache_ratio":0.9973568820374925,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":130560,"estimated_new_tokens":346,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":130906,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":130906,"score_linear":87576.0,"score_lmetric":141682},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":130560,"new_prefill":346,"score_linear":-130560.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":6656,"new_prefill":124250,"score_linear":-6656.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":130906,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":130906,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":27286,"ongoing_decode_tokens":27286,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2190,"cache_hit":0,"new_prefill":130906,"score_linear":27286.0,"score_lmetric":130906},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":130906,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-130560.0,"chosen_score_lmetric":0,"t_first_token":282569.727819456,"t_first_token_unix":1779713434.7417634,"t_done":282571.931110569,"t_done_unix":1779713436.9450536},{"request_id":"1243831:16:1446717:874","session_id":"1243831","input_length":29571,"t_proxy_recv":282571.917844584,"t_decision_unix":1779713436.931786,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":29184,"affinity_cache_ratio":0.9869128538094755,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":29184,"estimated_new_tokens":387,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":29571,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":159563,"ongoing_decode_tokens":87576,"pending_prefill_tokens":7987,"num_requests":2,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":11264,"new_prefill":18307,"score_linear":148299.0,"score_lmetric":52588},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":130906,"ongoing_decode_tokens":130906,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":29571,"score_linear":130906.0,"score_lmetric":29571},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":29571,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":29571,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":29571,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2194,"cache_hit":29184,"new_prefill":387,"score_linear":-29184.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":29571,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-29184.0,"chosen_score_lmetric":0,"t_first_token":282571.99848408,"t_first_token_unix":1779713437.0124278,"t_done":282572.425166707,"t_done_unix":1779713437.4391098},{"request_id":"1294611:15:1533841:949","session_id":"1294611","input_length":71987,"t_proxy_recv":282569.934170474,"t_decision_unix":1779713434.9481113,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":64000,"affinity_cache_ratio":0.8890494116993346,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":64000,"estimated_new_tokens":7987,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":71987,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":64000,"new_prefill":7987,"score_linear":23576.0,"score_lmetric":18763},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":130906,"ongoing_decode_tokens":130906,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":12800,"new_prefill":59187,"score_linear":118106.0,"score_lmetric":59187},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":12800,"new_prefill":59187,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":71987,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":71987,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":27286,"ongoing_decode_tokens":27286,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2190,"cache_hit":0,"new_prefill":71987,"score_linear":27286.0,"score_lmetric":71987},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":71987,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":23576.0,"chosen_score_lmetric":18763,"t_first_token":282575.229168246,"t_first_token_unix":1779713440.243112,"t_done":282579.770844898,"t_done_unix":1779713444.7847884},{"request_id":"1363943:32:1537767:950","session_id":"1363943","input_length":93368,"t_proxy_recv":282583.259440164,"t_decision_unix":1779713448.2733815,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":93184,"affinity_cache_ratio":0.9980293034015937,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":93184,"estimated_new_tokens":184,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":8704,"new_prefill":84664,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":87576,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1818,"cache_hit":24064,"new_prefill":69304,"score_linear":63512.0,"score_lmetric":69304},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2493,"cache_hit":0,"new_prefill":93368,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":4096,"new_prefill":89272,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":93368,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":93368,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2194,"cache_hit":93184,"new_prefill":184,"score_linear":-62023.0,"score_lmetric":184},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":89272,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-62023.0,"chosen_score_lmetric":184,"t_first_token":282583.490638487,"t_first_token_unix":1779713448.504582,"t_done":282585.895642318,"t_done_unix":1779713450.909586},{"request_id":"1253804:7:1526643:940","session_id":"1253804","input_length":87576,"t_proxy_recv":282545.922257904,"t_decision_unix":1779713410.936199,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":76800,"affinity_cache_ratio":0.8769525897506166,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":76800,"estimated_new_tokens":10776,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":87576,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":76800,"new_prefill":10776,"score_linear":-76800.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2492,"cache_hit":0,"new_prefill":87576,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":3584,"new_prefill":83992,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":87576,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2322,"cache_hit":0,"new_prefill":87576,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2084,"cache_hit":0,"new_prefill":87576,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":87576,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-76800.0,"chosen_score_lmetric":0,"t_first_token":282571.802219984,"t_first_token_unix":1779713436.8161638,"t_done":282591.581112637,"t_done_unix":1779713456.5950594},{"request_id":"1253804:8:1541192:953","session_id":"1253804","input_length":89116,"t_proxy_recv":282594.407068261,"t_decision_unix":1779713459.4210098,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":88576,"affinity_cache_ratio":0.9939404820683154,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":88576,"estimated_new_tokens":540,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":89116,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":88576,"new_prefill":540,"score_linear":-88576.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2493,"cache_hit":0,"new_prefill":89116,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":3584,"new_prefill":85532,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":89116,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":75693,"ongoing_decode_tokens":75693,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":89116,"score_linear":75693.0,"score_lmetric":89116},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":124757,"ongoing_decode_tokens":124757,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2194,"cache_hit":0,"new_prefill":89116,"score_linear":124757.0,"score_lmetric":178232},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":89116,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-88576.0,"chosen_score_lmetric":0,"t_first_token":282594.55034171,"t_first_token_unix":1779713459.5642853,"t_done":282594.968650521,"t_done_unix":1779713459.982595},{"request_id":"1268861:17:1540278:951","session_id":"1268861","input_length":75693,"t_proxy_recv":282591.341271451,"t_decision_unix":1779713456.3552127,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9402190427120077,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71168,"estimated_new_tokens":4525,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":75693,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":87576,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1818,"cache_hit":0,"new_prefill":75693,"score_linear":87576.0,"score_lmetric":75693},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2493,"cache_hit":0,"new_prefill":75693,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":75693,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":21504,"new_prefill":54189,"score_linear":-21504.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":71168,"new_prefill":4525,"score_linear":-71168.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2194,"cache_hit":0,"new_prefill":75693,"score_linear":31161.0,"score_lmetric":75693},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":75693,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-71168.0,"chosen_score_lmetric":0,"t_first_token":282593.515877667,"t_first_token_unix":1779713458.5298214,"t_done":282594.972681902,"t_done_unix":1779713459.986626},{"request_id":"1363943:33:1541065:952","session_id":"1363943","input_length":93596,"t_proxy_recv":282593.989662309,"t_decision_unix":1779713459.0036037,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":93184,"affinity_cache_ratio":0.9955981024830121,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":93184,"estimated_new_tokens":412,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":8704,"new_prefill":84892,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":24064,"new_prefill":69532,"score_linear":-24064.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2493,"cache_hit":0,"new_prefill":93596,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":4096,"new_prefill":89500,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":93596,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":75693,"ongoing_decode_tokens":75693,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":93596,"score_linear":75693.0,"score_lmetric":93596},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2194,"cache_hit":93184,"new_prefill":412,"score_linear":-62023.0,"score_lmetric":412},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":4096,"new_prefill":89500,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-62023.0,"chosen_score_lmetric":412,"t_first_token":282594.167401771,"t_first_token_unix":1779713459.181345,"t_done":282602.374597096,"t_done_unix":1779713467.388541},{"request_id":"1399948:23:1544108:954","session_id":"1399948","input_length":20031,"t_proxy_recv":282603.852298401,"t_decision_unix":1779713468.866239,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":18432,"affinity_cache_ratio":0.9201737307173881,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":18432,"estimated_new_tokens":1599,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":20031,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":20031,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2493,"cache_hit":0,"new_prefill":20031,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":20031,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":20031,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2347,"cache_hit":0,"new_prefill":20031,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2195,"cache_hit":18432,"new_prefill":1599,"score_linear":12729.0,"score_lmetric":1599},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":20031,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":12729.0,"chosen_score_lmetric":1599,"t_first_token":282604.143677781,"t_first_token_unix":1779713469.1576216,"t_done":282604.769314112,"t_done_unix":1779713469.783258},{"request_id":"1399948:24:1546940:955","session_id":"1399948","input_length":21809,"t_proxy_recv":282613.300663597,"t_decision_unix":1779713478.3146045,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":19968,"affinity_cache_ratio":0.9155853088174607,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":1841,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":21809,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":21809,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2493,"cache_hit":0,"new_prefill":21809,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":21809,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":21809,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2347,"cache_hit":0,"new_prefill":21809,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2198,"cache_hit":19968,"new_prefill":1841,"score_linear":11193.0,"score_lmetric":1841},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":21809,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":11193.0,"chosen_score_lmetric":1841,"t_first_token":282613.644588206,"t_first_token_unix":1779713478.6585321,"t_done":282614.016274811,"t_done_unix":1779713479.030218},{"request_id":"1243831:17:1453633:883","session_id":"1243831","input_length":31161,"t_proxy_recv":282572.435457429,"t_decision_unix":1779713437.449399,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":29184,"affinity_cache_ratio":0.9365553095215173,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":29184,"estimated_new_tokens":1977,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":31161,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":159563,"ongoing_decode_tokens":87576,"pending_prefill_tokens":7987,"num_requests":2,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":11264,"new_prefill":19897,"score_linear":148299.0,"score_lmetric":55768},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2493,"cache_hit":0,"new_prefill":31161,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":31161,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":31161,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":0,"new_prefill":31161,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2194,"cache_hit":29184,"new_prefill":1977,"score_linear":-29184.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":31161,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-29184.0,"chosen_score_lmetric":0,"t_first_token":282572.825301505,"t_first_token_unix":1779713437.839245,"t_done":282631.003294465,"t_done_unix":1779713496.0172384},{"request_id":"1399948:25:1553234:957","session_id":"1399948","input_length":22250,"t_proxy_recv":282633.833399301,"t_decision_unix":1779713498.8473403,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":21504,"affinity_cache_ratio":0.9664719101123596,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":21504,"estimated_new_tokens":746,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":22250,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":22250,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2493,"cache_hit":0,"new_prefill":22250,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":22250,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":22250,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":76017,"ongoing_decode_tokens":76017,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2347,"cache_hit":0,"new_prefill":22250,"score_linear":76017.0,"score_lmetric":22250},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2206,"cache_hit":21504,"new_prefill":746,"score_linear":-21504.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":22250,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-21504.0,"chosen_score_lmetric":0,"t_first_token":282633.979595725,"t_first_token_unix":1779713498.993539,"t_done":282634.528870084,"t_done_unix":1779713499.5428138},{"request_id":"1268861:18:1553083:956","session_id":"1268861","input_length":76017,"t_proxy_recv":282633.397927268,"t_decision_unix":1779713498.411868,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":75776,"affinity_cache_ratio":0.9968296565241985,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75776,"estimated_new_tokens":241,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2493,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":21504,"new_prefill":54513,"score_linear":-21504.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2347,"cache_hit":75776,"new_prefill":241,"score_linear":-75776.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2206,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75776.0,"chosen_score_lmetric":0,"t_first_token":282633.648876113,"t_first_token_unix":1779713498.6628199,"t_done":282634.678614139,"t_done_unix":1779713499.692558},{"request_id":"1399948:26:1557859:959","session_id":"1399948","input_length":23042,"t_proxy_recv":282649.809940518,"t_decision_unix":1779713514.8238814,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":22016,"affinity_cache_ratio":0.9554726152243729,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":22016,"estimated_new_tokens":1026,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2493,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2347,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2207,"cache_hit":22016,"new_prefill":1026,"score_linear":-22016.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-22016.0,"chosen_score_lmetric":0,"t_first_token":282650.015011227,"t_first_token_unix":1779713515.028955,"t_done":282650.300996832,"t_done_unix":1779713515.3149402},{"request_id":"1399948:27:1559017:960","session_id":"1399948","input_length":24139,"t_proxy_recv":282653.459962647,"t_decision_unix":1779713518.473904,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9544720162392808,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":23040,"estimated_new_tokens":1099,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2493,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2347,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2209,"cache_hit":23040,"new_prefill":1099,"score_linear":-23040.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-23040.0,"chosen_score_lmetric":0,"t_first_token":282653.725693337,"t_first_token_unix":1779713518.739637,"t_done":282654.682827511,"t_done_unix":1779713519.6967711},{"request_id":"1399948:28:1560859:961","session_id":"1399948","input_length":24386,"t_proxy_recv":282659.588303795,"t_decision_unix":1779713524.602245,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9867957024522267,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":322,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2493,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2347,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2211,"cache_hit":24064,"new_prefill":322,"score_linear":-24064.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":282659.670025743,"t_first_token_unix":1779713524.683969,"t_done":282660.940049469,"t_done_unix":1779713525.9539926},{"request_id":"1399948:29:1571125:963","session_id":"1399948","input_length":28781,"t_proxy_recv":282693.452261488,"t_decision_unix":1779713558.4662027,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":24576,"affinity_cache_ratio":0.8538966679406553,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24576,"estimated_new_tokens":4205,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":28781,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":28781,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":131413,"ongoing_decode_tokens":131413,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2493,"cache_hit":0,"new_prefill":28781,"score_linear":131413.0,"score_lmetric":28781},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":28781,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":28781,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2347,"cache_hit":0,"new_prefill":28781,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":24576,"new_prefill":4205,"score_linear":-24576.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":28781,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-24576.0,"chosen_score_lmetric":0,"t_first_token":282694.345485218,"t_first_token_unix":1779713559.3594286,"t_done":282694.614619854,"t_done_unix":1779713559.6285627},{"request_id":"1356885:6:1570780:962","session_id":"1356885","input_length":131413,"t_proxy_recv":282692.467383654,"t_decision_unix":1779713557.4813244,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":131072,"affinity_cache_ratio":0.9974051273466096,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":131072,"estimated_new_tokens":341,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":131413,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":131413,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2493,"cache_hit":131072,"new_prefill":341,"score_linear":-131072.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":6656,"new_prefill":124757,"score_linear":-6656.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":131413,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2347,"cache_hit":0,"new_prefill":131413,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":0,"new_prefill":131413,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":131413,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-131072.0,"chosen_score_lmetric":0,"t_first_token":282693.118500347,"t_first_token_unix":1779713558.132444,"t_done":282695.310272498,"t_done_unix":1779713560.3242161},{"request_id":"1268861:19:1578134:966","session_id":"1268861","input_length":76963,"t_proxy_recv":282717.30292783,"t_decision_unix":1779713582.316869,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":75776,"affinity_cache_ratio":0.9845770045346465,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75776,"estimated_new_tokens":1187,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":76963,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":76963,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":76963,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":76963,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":109083,"ongoing_decode_tokens":0,"pending_prefill_tokens":1051,"num_requests":1,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":21504,"new_prefill":55459,"score_linear":87579.0,"score_lmetric":56510},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2347,"cache_hit":75776,"new_prefill":1187,"score_linear":-75776.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":76963,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":76963,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75776.0,"chosen_score_lmetric":0,"t_first_token":282717.847920934,"t_first_token_unix":1779713582.8618648,"t_done":282718.61901079,"t_done_unix":1779713583.6329544},{"request_id":"1399948:30:1578524:967","session_id":"1399948","input_length":30773,"t_proxy_recv":282718.642109695,"t_decision_unix":1779713583.6560512,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":28672,"affinity_cache_ratio":0.9317258635817113,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":28672,"estimated_new_tokens":2101,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":109083,"ongoing_decode_tokens":0,"pending_prefill_tokens":1051,"num_requests":1,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":30773,"score_linear":109083.0,"score_lmetric":31824},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":28672,"new_prefill":2101,"score_linear":-28672.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-28672.0,"chosen_score_lmetric":0,"t_first_token":282719.14774299,"t_first_token_unix":1779713584.161687,"t_done":282719.448390135,"t_done_unix":1779713584.4623337},{"request_id":"1399948:31:1579418:968","session_id":"1399948","input_length":30925,"t_proxy_recv":282721.869443397,"t_decision_unix":1779713586.8833845,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":30720,"affinity_cache_ratio":0.9933710590137429,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":30720,"estimated_new_tokens":205,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":109083,"ongoing_decode_tokens":0,"pending_prefill_tokens":1051,"num_requests":1,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":30925,"score_linear":109083.0,"score_lmetric":31976},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2224,"cache_hit":30720,"new_prefill":205,"score_linear":-30720.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-30720.0,"chosen_score_lmetric":0,"t_first_token":282721.966811528,"t_first_token_unix":1779713586.9807549,"t_done":282723.301956989,"t_done_unix":1779713588.3159},{"request_id":"1356885:7:1580925:969","session_id":"1356885","input_length":131775,"t_proxy_recv":282727.183140489,"t_decision_unix":1779713592.1970813,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":131584,"affinity_cache_ratio":0.9985505596660975,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":131584,"estimated_new_tokens":191,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":131775,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":131775,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":131584,"new_prefill":191,"score_linear":-131584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":6656,"new_prefill":125119,"score_linear":-6656.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":109083,"ongoing_decode_tokens":0,"pending_prefill_tokens":1051,"num_requests":1,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":131775,"score_linear":109083.0,"score_lmetric":132826},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":0,"new_prefill":131775,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2224,"cache_hit":0,"new_prefill":131775,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":131775,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-131584.0,"chosen_score_lmetric":0,"t_first_token":282727.585940362,"t_first_token_unix":1779713592.5998845,"t_done":282729.605713158,"t_done_unix":1779713594.6196573},{"request_id":"1399948:32:1581424:970","session_id":"1399948","input_length":31231,"t_proxy_recv":282728.623970823,"t_decision_unix":1779713593.6379118,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":30720,"affinity_cache_ratio":0.9836380519355769,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":30720,"estimated_new_tokens":511,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":31231,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":31231,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":131775,"ongoing_decode_tokens":131775,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":31231,"score_linear":131775.0,"score_lmetric":31231},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":31231,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":109083,"ongoing_decode_tokens":0,"pending_prefill_tokens":1051,"num_requests":1,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":31231,"score_linear":109083.0,"score_lmetric":32282},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":0,"new_prefill":31231,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2224,"cache_hit":30720,"new_prefill":511,"score_linear":-30720.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":31231,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-30720.0,"chosen_score_lmetric":0,"t_first_token":282728.724486296,"t_first_token_unix":1779713593.7384298,"t_done":282729.728200273,"t_done_unix":1779713594.742144},{"request_id":"1399948:33:1583165:972","session_id":"1399948","input_length":31445,"t_proxy_recv":282734.531042058,"t_decision_unix":1779713599.5449831,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9932262680871363,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":31232,"estimated_new_tokens":213,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":109083,"ongoing_decode_tokens":0,"pending_prefill_tokens":1051,"num_requests":1,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":31445,"score_linear":109083.0,"score_lmetric":32496},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2225,"cache_hit":31232,"new_prefill":213,"score_linear":-31232.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-31232.0,"chosen_score_lmetric":0,"t_first_token":282734.616810202,"t_first_token_unix":1779713599.6307545,"t_done":282735.272717382,"t_done_unix":1779713600.286661},{"request_id":"1277428:12:1586139:973","session_id":"1277428","input_length":47660,"t_proxy_recv":282744.668204646,"t_decision_unix":1779713609.6821456,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":46080,"affinity_cache_ratio":0.9668485102811583,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":46080,"estimated_new_tokens":1580,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":47660,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":4608,"new_prefill":43052,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":47660,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":46080,"new_prefill":1580,"score_linear":-46080.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":109083,"ongoing_decode_tokens":0,"pending_prefill_tokens":1051,"num_requests":1,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":47660,"score_linear":109083.0,"score_lmetric":48711},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":0,"new_prefill":47660,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2225,"cache_hit":0,"new_prefill":47660,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":47660,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-46080.0,"chosen_score_lmetric":0,"t_first_token":282745.129972785,"t_first_token_unix":1779713610.1439176,"t_done":282745.741410093,"t_done_unix":1779713610.7553535},{"request_id":"1399948:34:1586232:974","session_id":"1399948","input_length":32143,"t_proxy_recv":282745.0180223,"t_decision_unix":1779713610.0319633,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":31232,"affinity_cache_ratio":0.97165790374265,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":31232,"estimated_new_tokens":911,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":32143,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":32143,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":32143,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":47660,"ongoing_decode_tokens":0,"pending_prefill_tokens":1580,"num_requests":1,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":32143,"score_linear":47660.0,"score_lmetric":33723},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":109083,"ongoing_decode_tokens":0,"pending_prefill_tokens":1051,"num_requests":1,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":32143,"score_linear":109083.0,"score_lmetric":33194},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":0,"new_prefill":32143,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2225,"cache_hit":31232,"new_prefill":911,"score_linear":-31232.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":32143,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-31232.0,"chosen_score_lmetric":0,"t_first_token":282745.218118873,"t_first_token_unix":1779713610.232063,"t_done":282746.118559358,"t_done_unix":1779713611.1325026},{"request_id":"1270606:4:1578042:965","session_id":"1270606","input_length":109083,"t_proxy_recv":282717.077910916,"t_decision_unix":1779713582.0918517,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":108032,"affinity_cache_ratio":0.9903651348056067,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":108032,"estimated_new_tokens":1051,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2121,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":108032,"new_prefill":1051,"score_linear":-108032.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2347,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":107547,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-108032.0,"chosen_score_lmetric":0,"t_first_token":282755.763314912,"t_first_token_unix":1779713620.7772596,"t_done":282756.196253414,"t_done_unix":1779713621.2101972},{"request_id":"1270606:5:1581455:971","session_id":"1270606","input_length":111387,"t_proxy_recv":282756.235915556,"t_decision_unix":1779713621.2498565,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":109056,"affinity_cache_ratio":0.9790729618357618,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":109056,"estimated_new_tokens":2331,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":111387,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":111387,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":111387,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2124,"cache_hit":0,"new_prefill":111387,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2217,"cache_hit":109056,"new_prefill":2331,"score_linear":-109056.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":0,"new_prefill":111387,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2227,"cache_hit":0,"new_prefill":111387,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":109851,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-109056.0,"chosen_score_lmetric":0,"t_first_token":282757.99634228,"t_first_token_unix":1779713623.0102887,"t_done":282760.842930217,"t_done_unix":1779713625.856874},{"request_id":"1294611:15:1590202:975","session_id":"1294611","input_length":75249,"t_proxy_recv":282758.205244115,"t_decision_unix":1779713623.2191849,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":72192,"affinity_cache_ratio":0.9593748754136268,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":72192,"estimated_new_tokens":3057,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":75249,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":72192,"new_prefill":3057,"score_linear":-72192.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":12800,"new_prefill":62449,"score_linear":-12800.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2124,"cache_hit":12800,"new_prefill":62449,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":111387,"ongoing_decode_tokens":111387,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2217,"cache_hit":0,"new_prefill":75249,"score_linear":111387.0,"score_lmetric":75249},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":0,"new_prefill":75249,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2227,"cache_hit":0,"new_prefill":75249,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":75249,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-72192.0,"chosen_score_lmetric":0,"t_first_token":282759.72574892,"t_first_token_unix":1779713624.7396922,"t_done":282762.44825865,"t_done_unix":1779713627.4622025},{"request_id":"1277428:13:1593499:976","session_id":"1277428","input_length":49316,"t_proxy_recv":282769.848485185,"t_decision_unix":1779713634.8624258,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":47616,"affinity_cache_ratio":0.9655284289074539,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":47616,"estimated_new_tokens":1700,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":49316,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1847,"cache_hit":4608,"new_prefill":44708,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":49316,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2124,"cache_hit":47616,"new_prefill":1700,"score_linear":-47616.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":49316,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":0,"new_prefill":49316,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2227,"cache_hit":0,"new_prefill":49316,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":49316,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-47616.0,"chosen_score_lmetric":0,"t_first_token":282770.497307707,"t_first_token_unix":1779713635.511252,"t_done":282772.02456507,"t_done_unix":1779713637.038509},{"request_id":"1294611:15:1594038:977","session_id":"1294611","input_length":75584,"t_proxy_recv":282771.56917986,"t_decision_unix":1779713636.5831206,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":75264,"affinity_cache_ratio":0.995766299745978,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75264,"estimated_new_tokens":320,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":75584,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1847,"cache_hit":75264,"new_prefill":320,"score_linear":-75264.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":12800,"new_prefill":62784,"score_linear":-12800.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49316,"ongoing_decode_tokens":49316,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2124,"cache_hit":12800,"new_prefill":62784,"score_linear":36516.0,"score_lmetric":62784},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":75584,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":0,"new_prefill":75584,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2227,"cache_hit":0,"new_prefill":75584,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":75584,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75264.0,"chosen_score_lmetric":0,"t_first_token":282771.696471548,"t_first_token_unix":1779713636.7104154,"t_done":282772.810816332,"t_done_unix":1779713637.82476},{"request_id":"1268861:20:1595141:978","session_id":"1268861","input_length":77218,"t_proxy_recv":282775.548579168,"t_decision_unix":1779713640.56252,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":76800,"affinity_cache_ratio":0.9945867543836929,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":76800,"estimated_new_tokens":418,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1847,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":21504,"new_prefill":55714,"score_linear":-21504.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":76800,"new_prefill":418,"score_linear":-76800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2227,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-76800.0,"chosen_score_lmetric":0,"t_first_token":282775.796761491,"t_first_token_unix":1779713640.8107057,"t_done":282776.325911464,"t_done_unix":1779713641.339855},{"request_id":"1399948:35:1602048:980","session_id":"1399948","input_length":33533,"t_proxy_recv":282798.931918048,"t_decision_unix":1779713663.945859,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":32256,"affinity_cache_ratio":0.9619181105179972,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":32256,"estimated_new_tokens":1277,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":33533,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75716,"ongoing_decode_tokens":75716,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1847,"cache_hit":0,"new_prefill":33533,"score_linear":75716.0,"score_lmetric":33533},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":33533,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":0,"new_prefill":33533,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":33533,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":0,"new_prefill":33533,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2227,"cache_hit":32256,"new_prefill":1277,"score_linear":-32256.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":33533,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-32256.0,"chosen_score_lmetric":0,"t_first_token":282799.343178739,"t_first_token_unix":1779713664.357122,"t_done":282800.362156292,"t_done_unix":1779713665.3760998},{"request_id":"1253804:9:1605110:981","session_id":"1253804","input_length":90656,"t_proxy_recv":282808.912170627,"t_decision_unix":1779713673.9261117,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":89088,"affinity_cache_ratio":0.9827038475114719,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":89088,"estimated_new_tokens":1568,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":90656,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75716,"ongoing_decode_tokens":75716,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1847,"cache_hit":89088,"new_prefill":1568,"score_linear":-13372.0,"score_lmetric":1568},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":90656,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":3584,"new_prefill":87072,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":90656,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":0,"new_prefill":90656,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2229,"cache_hit":0,"new_prefill":90656,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":90656,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-13372.0,"chosen_score_lmetric":1568,"t_first_token":282809.887798029,"t_first_token_unix":1779713674.9017422,"t_done":282815.200533452,"t_done_unix":1779713680.214477},{"request_id":"1294611:15:1596163:979","session_id":"1294611","input_length":75716,"t_proxy_recv":282778.732507856,"t_decision_unix":1779713643.746449,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":75264,"affinity_cache_ratio":0.9940303238417243,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75264,"estimated_new_tokens":452,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":75716,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1847,"cache_hit":75264,"new_prefill":452,"score_linear":-75264.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":12800,"new_prefill":62916,"score_linear":-12800.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":12800,"new_prefill":62916,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":75716,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":0,"new_prefill":75716,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2227,"cache_hit":0,"new_prefill":75716,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":75716,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75264.0,"chosen_score_lmetric":0,"t_first_token":282778.853098993,"t_first_token_unix":1779713643.867043,"t_done":282820.228663321,"t_done_unix":1779713685.2426116},{"request_id":"1268861:21:1609831:982","session_id":"1268861","input_length":77293,"t_proxy_recv":282824.654568868,"t_decision_unix":1779713689.6685102,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":76800,"affinity_cache_ratio":0.9936216733727504,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":76800,"estimated_new_tokens":493,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1852,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":21504,"new_prefill":55789,"score_linear":-21504.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":76800,"new_prefill":493,"score_linear":-76800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2229,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-76800.0,"chosen_score_lmetric":0,"t_first_token":282824.765719237,"t_first_token_unix":1779713689.7796626,"t_done":282826.365307474,"t_done_unix":1779713691.3792508},{"request_id":"1253804:10:1611505:983","session_id":"1253804","input_length":91103,"t_proxy_recv":282830.133538204,"t_decision_unix":1779713695.1474793,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":90624,"affinity_cache_ratio":0.9947422148557128,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":90624,"estimated_new_tokens":479,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":91103,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1852,"cache_hit":90624,"new_prefill":479,"score_linear":-90624.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":91103,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":3584,"new_prefill":87519,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":91103,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2350,"cache_hit":0,"new_prefill":91103,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2229,"cache_hit":0,"new_prefill":91103,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":91103,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-90624.0,"chosen_score_lmetric":0,"t_first_token":282830.309576727,"t_first_token_unix":1779713695.3235202,"t_done":282830.436954943,"t_done_unix":1779713695.4508984},{"request_id":"1399948:36:1612467:984","session_id":"1399948","input_length":34773,"t_proxy_recv":282833.240387558,"t_decision_unix":1779713698.2543285,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":33280,"affinity_cache_ratio":0.9570643890374716,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":33280,"estimated_new_tokens":1493,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1852,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2350,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2229,"cache_hit":33280,"new_prefill":1493,"score_linear":-33280.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-33280.0,"chosen_score_lmetric":0,"t_first_token":282833.593106538,"t_first_token_unix":1779713698.6070502,"t_done":282834.139833841,"t_done_unix":1779713699.153777},{"request_id":"1253804:11:1613030:985","session_id":"1253804","input_length":91131,"t_proxy_recv":282835.46928641,"t_decision_unix":1779713700.4832268,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":90624,"affinity_cache_ratio":0.9944365803074695,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":90624,"estimated_new_tokens":507,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":91131,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1852,"cache_hit":90624,"new_prefill":507,"score_linear":-90624.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":91131,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":3584,"new_prefill":87547,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":91131,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2350,"cache_hit":0,"new_prefill":91131,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2232,"cache_hit":0,"new_prefill":91131,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":91131,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-90624.0,"chosen_score_lmetric":0,"t_first_token":282835.592895576,"t_first_token_unix":1779713700.6068401,"t_done":282837.67841805,"t_done_unix":1779713702.6923637},{"request_id":"1399948:37:1613628:986","session_id":"1399948","input_length":35308,"t_proxy_recv":282837.316693569,"t_decision_unix":1779713702.3306346,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9860654809108418,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":492,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":91131,"ongoing_decode_tokens":91131,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1852,"cache_hit":0,"new_prefill":35308,"score_linear":91131.0,"score_lmetric":35308},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2350,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2232,"cache_hit":34816,"new_prefill":492,"score_linear":-34816.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-34816.0,"chosen_score_lmetric":0,"t_first_token":282837.527218502,"t_first_token_unix":1779713702.5411615,"t_done":282838.667123774,"t_done_unix":1779713703.6810672},{"request_id":"1399948:38:1616890:987","session_id":"1399948","input_length":35581,"t_proxy_recv":282848.498688916,"t_decision_unix":1779713713.5126297,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9928894634776988,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35328,"estimated_new_tokens":253,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1853,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2350,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2233,"cache_hit":35328,"new_prefill":253,"score_linear":-35328.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-35328.0,"chosen_score_lmetric":0,"t_first_token":282848.608183398,"t_first_token_unix":1779713713.6221268,"t_done":282849.127377914,"t_done_unix":1779713714.1413217},{"request_id":"1399948:39:1618250:989","session_id":"1399948","input_length":35682,"t_proxy_recv":282853.19409016,"t_decision_unix":1779713718.208031,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9900790314444258,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35328,"estimated_new_tokens":354,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1853,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2350,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2233,"cache_hit":35328,"new_prefill":354,"score_linear":-35328.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-35328.0,"chosen_score_lmetric":0,"t_first_token":282853.265734243,"t_first_token_unix":1779713718.2796779,"t_done":282853.708525803,"t_done_unix":1779713718.722469},{"request_id":"1399948:40:1620970:991","session_id":"1399948","input_length":35958,"t_proxy_recv":282862.188962159,"t_decision_unix":1779713727.2029028,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9824795594860671,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35328,"estimated_new_tokens":630,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":35958,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1853,"cache_hit":0,"new_prefill":35958,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":35958,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":0,"new_prefill":35958,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":35958,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78439,"ongoing_decode_tokens":0,"pending_prefill_tokens":1127,"num_requests":1,"active_p_offloads":0,"cached_blocks":2350,"cache_hit":0,"new_prefill":35958,"score_linear":78439.0,"score_lmetric":37085},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2233,"cache_hit":35328,"new_prefill":630,"score_linear":-35328.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":35958,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-35328.0,"chosen_score_lmetric":0,"t_first_token":282862.325210274,"t_first_token_unix":1779713727.3391542,"t_done":282862.689877516,"t_done_unix":1779713727.7038207},{"request_id":"1268861:22:1620830:990","session_id":"1268861","input_length":78439,"t_proxy_recv":282861.722287059,"t_decision_unix":1779713726.7362278,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":77312,"affinity_cache_ratio":0.9856321472736777,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":77312,"estimated_new_tokens":1127,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1853,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":21504,"new_prefill":56935,"score_linear":-21504.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2350,"cache_hit":77312,"new_prefill":1127,"score_linear":-77312.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2233,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-77312.0,"chosen_score_lmetric":0,"t_first_token":282862.413548703,"t_first_token_unix":1779713727.4274926,"t_done":282863.686234141,"t_done_unix":1779713728.7001774},{"request_id":"1253804:12:1621957:992","session_id":"1253804","input_length":91389,"t_proxy_recv":282865.627532811,"t_decision_unix":1779713730.6414733,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":91136,"affinity_cache_ratio":0.9972316143080677,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":91136,"estimated_new_tokens":253,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":91389,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1853,"cache_hit":91136,"new_prefill":253,"score_linear":-91136.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":91389,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":3584,"new_prefill":87805,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":91389,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2352,"cache_hit":0,"new_prefill":91389,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":0,"new_prefill":91389,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":91389,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-91136.0,"chosen_score_lmetric":0,"t_first_token":282865.816777638,"t_first_token_unix":1779713730.8307214,"t_done":282866.10178974,"t_done_unix":1779713731.115733},{"request_id":"1399948:41:1624143:993","session_id":"1399948","input_length":36168,"t_proxy_recv":282872.991994844,"t_decision_unix":1779713738.005936,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":35840,"affinity_cache_ratio":0.990931209909312,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35840,"estimated_new_tokens":328,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1853,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2352,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":35840,"new_prefill":328,"score_linear":-35840.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-35840.0,"chosen_score_lmetric":0,"t_first_token":282873.124958924,"t_first_token_unix":1779713738.1389022,"t_done":282873.795061703,"t_done_unix":1779713738.8090053},{"request_id":"1253804:13:1624613:994","session_id":"1253804","input_length":91440,"t_proxy_recv":282874.566736895,"t_decision_unix":1779713739.5806782,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":91136,"affinity_cache_ratio":0.9966754155730534,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":91136,"estimated_new_tokens":304,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":91440,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1853,"cache_hit":91136,"new_prefill":304,"score_linear":-91136.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":91440,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":3584,"new_prefill":87856,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":91440,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2352,"cache_hit":0,"new_prefill":91440,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":0,"new_prefill":91440,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":91440,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-91136.0,"chosen_score_lmetric":0,"t_first_token":282874.699662325,"t_first_token_unix":1779713739.7136059,"t_done":282875.699271945,"t_done_unix":1779713740.7132154},{"request_id":"1253804:14:1626575:995","session_id":"1253804","input_length":92367,"t_proxy_recv":282881.081295768,"t_decision_unix":1779713746.0952368,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":91136,"affinity_cache_ratio":0.9866727294380028,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":91136,"estimated_new_tokens":1231,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":92367,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1853,"cache_hit":91136,"new_prefill":1231,"score_linear":-91136.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":92367,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":3584,"new_prefill":88783,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":92367,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2352,"cache_hit":0,"new_prefill":92367,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":0,"new_prefill":92367,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":92367,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-91136.0,"chosen_score_lmetric":0,"t_first_token":282881.724657092,"t_first_token_unix":1779713746.7386007,"t_done":282882.531714696,"t_done_unix":1779713747.5456583},{"request_id":"1253804:15:1628485:997","session_id":"1253804","input_length":92701,"t_proxy_recv":282887.404959164,"t_decision_unix":1779713752.4189007,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":92160,"affinity_cache_ratio":0.9941640327504557,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":92160,"estimated_new_tokens":541,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":92701,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1855,"cache_hit":92160,"new_prefill":541,"score_linear":-92160.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":92701,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":3584,"new_prefill":89117,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":92701,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2352,"cache_hit":0,"new_prefill":92701,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":36984,"ongoing_decode_tokens":0,"pending_prefill_tokens":1144,"num_requests":1,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":0,"new_prefill":92701,"score_linear":36984.0,"score_lmetric":93845},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":92701,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-92160.0,"chosen_score_lmetric":0,"t_first_token":282887.697541204,"t_first_token_unix":1779713752.7114844,"t_done":282888.073447736,"t_done_unix":1779713753.0873907},{"request_id":"1399948:42:1628429:996","session_id":"1399948","input_length":36984,"t_proxy_recv":282887.184750618,"t_decision_unix":1779713752.1986916,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":35840,"affinity_cache_ratio":0.9690677049534934,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35840,"estimated_new_tokens":1144,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1855,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2352,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":35840,"new_prefill":1144,"score_linear":-35840.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-35840.0,"chosen_score_lmetric":0,"t_first_token":282887.474954122,"t_first_token_unix":1779713752.4888976,"t_done":282888.686482763,"t_done_unix":1779713753.7004263},{"request_id":"1388507:3:1629072:998","session_id":"1388507","input_length":79074,"t_proxy_recv":282889.493096136,"t_decision_unix":1779713754.5070372,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":75776,"affinity_cache_ratio":0.9582922325922554,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75776,"estimated_new_tokens":3298,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1856,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":75776,"new_prefill":3298,"score_linear":-75776.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2352,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75776.0,"chosen_score_lmetric":0,"t_first_token":282891.212760443,"t_first_token_unix":1779713756.2267041,"t_done":282891.766318742,"t_done_unix":1779713756.7802615},{"request_id":"1253804:16:1629713:999","session_id":"1253804","input_length":92953,"t_proxy_recv":282891.45491809,"t_decision_unix":1779713756.4688578,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":92672,"affinity_cache_ratio":0.9969769668542167,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":92672,"estimated_new_tokens":281,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":92953,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1856,"cache_hit":92672,"new_prefill":281,"score_linear":-92672.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79074,"ongoing_decode_tokens":79074,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":0,"new_prefill":92953,"score_linear":79074.0,"score_lmetric":92953},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":3584,"new_prefill":89369,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":92953,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2352,"cache_hit":0,"new_prefill":92953,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":0,"new_prefill":92953,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":92953,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-92672.0,"chosen_score_lmetric":0,"t_first_token":282891.747533246,"t_first_token_unix":1779713756.7614765,"t_done":282892.728101462,"t_done_unix":1779713757.7420447},{"request_id":"1268861:23:1631587:1001","session_id":"1268861","input_length":79693,"t_proxy_recv":282897.454456225,"t_decision_unix":1779713762.4683976,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":78336,"affinity_cache_ratio":0.9829721556472965,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":78336,"estimated_new_tokens":1357,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":79693,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":79890,"ongoing_decode_tokens":79890,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1856,"cache_hit":0,"new_prefill":79693,"score_linear":79890.0,"score_lmetric":79693},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":79693,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":0,"new_prefill":79693,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":21504,"new_prefill":58189,"score_linear":-21504.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2352,"cache_hit":78336,"new_prefill":1357,"score_linear":-78336.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":0,"new_prefill":79693,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":79693,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-78336.0,"chosen_score_lmetric":0,"t_first_token":282898.160438632,"t_first_token_unix":1779713763.174382,"t_done":282898.800782941,"t_done_unix":1779713763.814726},{"request_id":"1253804:17:1631602:1002","session_id":"1253804","input_length":93219,"t_proxy_recv":282897.528273566,"t_decision_unix":1779713762.5422149,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":92672,"affinity_cache_ratio":0.9941320975337645,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":92672,"estimated_new_tokens":547,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":93219,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":79890,"ongoing_decode_tokens":79890,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1856,"cache_hit":92672,"new_prefill":547,"score_linear":-12782.0,"score_lmetric":547},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":93219,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":3584,"new_prefill":89635,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":93219,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":79693,"ongoing_decode_tokens":0,"pending_prefill_tokens":1357,"num_requests":1,"active_p_offloads":0,"cached_blocks":2352,"cache_hit":0,"new_prefill":93219,"score_linear":79693.0,"score_lmetric":94576},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":0,"new_prefill":93219,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":93219,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-12782.0,"chosen_score_lmetric":547,"t_first_token":282897.820061094,"t_first_token_unix":1779713762.8340054,"t_done":282898.96713768,"t_done_unix":1779713763.9810822},{"request_id":"1294611:15:1631116:1000","session_id":"1294611","input_length":79890,"t_proxy_recv":282896.038930752,"t_decision_unix":1779713761.0528717,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":79360,"affinity_cache_ratio":0.9933658780823632,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":79360,"estimated_new_tokens":530,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":79890,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1856,"cache_hit":79360,"new_prefill":530,"score_linear":-79360.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":12800,"new_prefill":67090,"score_linear":-12800.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":12800,"new_prefill":67090,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":79890,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2352,"cache_hit":0,"new_prefill":79890,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":0,"new_prefill":79890,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":79890,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-79360.0,"chosen_score_lmetric":0,"t_first_token":282896.218894199,"t_first_token_unix":1779713761.232838,"t_done":282899.614586016,"t_done_unix":1779713764.6285312},{"request_id":"1253804:18:1633257:1003","session_id":"1253804","input_length":93509,"t_proxy_recv":282902.983252945,"t_decision_unix":1779713767.9971943,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":93184,"affinity_cache_ratio":0.9965243987209788,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":93184,"estimated_new_tokens":325,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":93509,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1858,"cache_hit":93184,"new_prefill":325,"score_linear":-93184.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":93509,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":3584,"new_prefill":89925,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":93509,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2354,"cache_hit":0,"new_prefill":93509,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":0,"new_prefill":93509,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":93509,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-93184.0,"chosen_score_lmetric":0,"t_first_token":282903.277803307,"t_first_token_unix":1779713768.291747,"t_done":282904.166231209,"t_done_unix":1779713769.1801746},{"request_id":"1294611:15:1634779:1004","session_id":"1294611","input_length":80221,"t_proxy_recv":282908.14880014,"t_decision_unix":1779713773.1627412,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":79872,"affinity_cache_ratio":0.995649518205956,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":79872,"estimated_new_tokens":349,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":80221,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1858,"cache_hit":79872,"new_prefill":349,"score_linear":-79872.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":12800,"new_prefill":67421,"score_linear":-12800.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":12800,"new_prefill":67421,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":80221,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2354,"cache_hit":0,"new_prefill":80221,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":0,"new_prefill":80221,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":80221,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-79872.0,"chosen_score_lmetric":0,"t_first_token":282908.280234777,"t_first_token_unix":1779713773.2941782,"t_done":282909.088712042,"t_done_unix":1779713774.1026556},{"request_id":"1253804:19:1634903:1005","session_id":"1253804","input_length":93664,"t_proxy_recv":282908.523621195,"t_decision_unix":1779713773.5375624,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":93184,"affinity_cache_ratio":0.9948752989408951,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":93184,"estimated_new_tokens":480,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":93664,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80221,"ongoing_decode_tokens":80221,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1858,"cache_hit":93184,"new_prefill":480,"score_linear":-12963.0,"score_lmetric":480},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":93664,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":3584,"new_prefill":90080,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":93664,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2354,"cache_hit":0,"new_prefill":93664,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":0,"new_prefill":93664,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":93664,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-12963.0,"chosen_score_lmetric":480,"t_first_token":282908.723732908,"t_first_token_unix":1779713773.7376764,"t_done":282912.87621202,"t_done_unix":1779713777.8901553},{"request_id":"1253804:20:1638889:1007","session_id":"1253804","input_length":94111,"t_proxy_recv":282921.701670255,"t_decision_unix":1779713786.7156112,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":93696,"affinity_cache_ratio":0.9955903135658956,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":93696,"estimated_new_tokens":415,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":94111,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80717,"ongoing_decode_tokens":80717,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":93696,"new_prefill":415,"score_linear":-12979.0,"score_lmetric":415},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":94111,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":3584,"new_prefill":90527,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":94111,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2354,"cache_hit":0,"new_prefill":94111,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":0,"new_prefill":94111,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":94111,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-12979.0,"chosen_score_lmetric":415,"t_first_token":282921.899986001,"t_first_token_unix":1779713786.91393,"t_done":282923.148642601,"t_done_unix":1779713788.1625862},{"request_id":"1258908:2:1639411:1008","session_id":"1258908","input_length":9804,"t_proxy_recv":282923.501236038,"t_decision_unix":1779713788.5151772,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":9216,"affinity_cache_ratio":0.9400244798041616,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9216,"estimated_new_tokens":588,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80717,"ongoing_decode_tokens":80717,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":9804,"score_linear":80717.0,"score_lmetric":9804},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":9216,"new_prefill":588,"score_linear":-9216.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2354,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9216.0,"chosen_score_lmetric":0,"t_first_token":282924.254168403,"t_first_token_unix":1779713789.2681124,"t_done":282925.827138024,"t_done_unix":1779713790.8410811},{"request_id":"1268861:24:1643502:1011","session_id":"1268861","input_length":80019,"t_proxy_recv":282937.328435959,"t_decision_unix":1779713802.3423772,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":79360,"affinity_cache_ratio":0.9917644559417138,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":79360,"estimated_new_tokens":659,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":80019,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":174986,"ongoing_decode_tokens":174986,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":80019,"score_linear":174986.0,"score_lmetric":160038},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":80019,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":80019,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":21504,"new_prefill":58515,"score_linear":-21504.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2354,"cache_hit":79360,"new_prefill":659,"score_linear":-79360.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":38102,"ongoing_decode_tokens":0,"pending_prefill_tokens":1238,"num_requests":1,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":0,"new_prefill":80019,"score_linear":38102.0,"score_lmetric":81257},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":80019,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-79360.0,"chosen_score_lmetric":0,"t_first_token":282937.588039274,"t_first_token_unix":1779713802.601983,"t_done":282938.050094119,"t_done_unix":1779713803.0640376},{"request_id":"1399948:43:1643413:1010","session_id":"1399948","input_length":38102,"t_proxy_recv":282936.979443261,"t_decision_unix":1779713801.9933841,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":36864,"affinity_cache_ratio":0.9675082672825573,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":36864,"estimated_new_tokens":1238,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":38102,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":174986,"ongoing_decode_tokens":174986,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":38102,"score_linear":174986.0,"score_lmetric":76204},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":38102,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":38102,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":38102,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2354,"cache_hit":0,"new_prefill":38102,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":36864,"new_prefill":1238,"score_linear":-36864.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":38102,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-36864.0,"chosen_score_lmetric":0,"t_first_token":282937.354622434,"t_first_token_unix":1779713802.3685656,"t_done":282938.137688571,"t_done_unix":1779713803.1516318},{"request_id":"1399948:44:1644970:1012","session_id":"1399948","input_length":38308,"t_proxy_recv":282942.104602277,"t_decision_unix":1779713807.1185431,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":37888,"affinity_cache_ratio":0.9890362326407017,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":37888,"estimated_new_tokens":420,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":174986,"ongoing_decode_tokens":174986,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":0,"new_prefill":38308,"score_linear":174986.0,"score_lmetric":76616},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2355,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":37888,"new_prefill":420,"score_linear":-37888.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-37888.0,"chosen_score_lmetric":0,"t_first_token":282942.217150751,"t_first_token_unix":1779713807.2310944,"t_done":282943.739196459,"t_done_unix":1779713808.75314},{"request_id":"1294611:15:1636645:1006","session_id":"1294611","input_length":80717,"t_proxy_recv":282914.192107451,"t_decision_unix":1779713779.2060487,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":79872,"affinity_cache_ratio":0.9895313254952488,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":79872,"estimated_new_tokens":845,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":80717,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":79872,"new_prefill":845,"score_linear":-79872.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":12800,"new_prefill":67917,"score_linear":-12800.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":12800,"new_prefill":67917,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":80717,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2354,"cache_hit":0,"new_prefill":80717,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":0,"new_prefill":80717,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":80717,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-79872.0,"chosen_score_lmetric":0,"t_first_token":282914.605697857,"t_first_token_unix":1779713779.6196415,"t_done":282945.097232096,"t_done_unix":1779713810.1111755},{"request_id":"1253804:21:1640487:1009","session_id":"1253804","input_length":94269,"t_proxy_recv":282926.990233462,"t_decision_unix":1779713792.0041747,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":93696,"affinity_cache_ratio":0.9939216497470006,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":93696,"estimated_new_tokens":573,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":94269,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80717,"ongoing_decode_tokens":80717,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1859,"cache_hit":93696,"new_prefill":573,"score_linear":-12979.0,"score_lmetric":573},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":94269,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":90685,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":94269,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2354,"cache_hit":0,"new_prefill":94269,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":0,"new_prefill":94269,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":94269,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-12979.0,"chosen_score_lmetric":573,"t_first_token":282927.195989047,"t_first_token_unix":1779713792.2099328,"t_done":282947.278240431,"t_done_unix":1779713812.292184},{"request_id":"1253804:22:1650001:1013","session_id":"1253804","input_length":95715,"t_proxy_recv":282958.694710156,"t_decision_unix":1779713823.7086492,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":95232,"affinity_cache_ratio":0.9949537690017238,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":95232,"estimated_new_tokens":483,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":95715,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1864,"cache_hit":95232,"new_prefill":483,"score_linear":-95232.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":95715,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":92131,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":95715,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2355,"cache_hit":0,"new_prefill":95715,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":0,"new_prefill":95715,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":95715,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-95232.0,"chosen_score_lmetric":0,"t_first_token":282958.895259302,"t_first_token_unix":1779713823.9092035,"t_done":282959.382679442,"t_done_unix":1779713824.3966234},{"request_id":"1399948:45:1650679:1014","session_id":"1399948","input_length":38758,"t_proxy_recv":282961.145678002,"t_decision_unix":1779713826.1596193,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":38400,"affinity_cache_ratio":0.9907631972754012,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":38400,"estimated_new_tokens":358,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1865,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2355,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":38400,"new_prefill":358,"score_linear":-38400.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-38400.0,"chosen_score_lmetric":0,"t_first_token":282961.299584869,"t_first_token_unix":1779713826.313528,"t_done":282961.887674404,"t_done_unix":1779713826.9016178},{"request_id":"1253804:23:1651448:1015","session_id":"1253804","input_length":96944,"t_proxy_recv":282963.445059011,"t_decision_unix":1779713828.459,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":95744,"affinity_cache_ratio":0.9876217197557353,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":95744,"estimated_new_tokens":1200,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":96944,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1865,"cache_hit":95744,"new_prefill":1200,"score_linear":-95744.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":96944,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":93360,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":96944,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2355,"cache_hit":0,"new_prefill":96944,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":0,"new_prefill":96944,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":96944,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-95744.0,"chosen_score_lmetric":0,"t_first_token":282964.289312956,"t_first_token_unix":1779713829.3032568,"t_done":282964.69651591,"t_done_unix":1779713829.7104602},{"request_id":"1399948:46:1658183:1018","session_id":"1399948","input_length":39837,"t_proxy_recv":282986.282786418,"t_decision_unix":1779713851.296728,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":38400,"affinity_cache_ratio":0.9639280066270051,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":38400,"estimated_new_tokens":1437,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":39837,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":83204,"ongoing_decode_tokens":83204,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1867,"cache_hit":0,"new_prefill":39837,"score_linear":83204.0,"score_lmetric":39837},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":39837,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":39837,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":39837,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":80406,"ongoing_decode_tokens":80406,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2355,"cache_hit":0,"new_prefill":39837,"score_linear":80406.0,"score_lmetric":39837},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":38400,"new_prefill":1437,"score_linear":-38400.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":39837,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-38400.0,"chosen_score_lmetric":0,"t_first_token":282986.677816108,"t_first_token_unix":1779713851.6917596,"t_done":282987.525949999,"t_done_unix":1779713852.5398932},{"request_id":"1253804:24:1659020:1019","session_id":"1253804","input_length":97769,"t_proxy_recv":282989.175963414,"t_decision_unix":1779713854.1899047,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":96768,"affinity_cache_ratio":0.9897615808691916,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":96768,"estimated_new_tokens":1001,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":97769,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":83204,"ongoing_decode_tokens":83204,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1867,"cache_hit":96768,"new_prefill":1001,"score_linear":-13564.0,"score_lmetric":1001},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":97769,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":94185,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":97769,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":80406,"ongoing_decode_tokens":80406,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2355,"cache_hit":0,"new_prefill":97769,"score_linear":80406.0,"score_lmetric":97769},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":97769,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":97769,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-13564.0,"chosen_score_lmetric":1001,"t_first_token":282989.848298761,"t_first_token_unix":1779713854.8622422,"t_done":282990.475909465,"t_done_unix":1779713855.4898527},{"request_id":"1268861:25:1653400:1016","session_id":"1268861","input_length":80406,"t_proxy_recv":282969.885868607,"t_decision_unix":1779713834.8998091,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":79872,"affinity_cache_ratio":0.9933587045742855,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":79872,"estimated_new_tokens":534,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1867,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":21504,"new_prefill":58902,"score_linear":-21504.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2355,"cache_hit":79872,"new_prefill":534,"score_linear":-79872.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-79872.0,"chosen_score_lmetric":0,"t_first_token":282970.312743292,"t_first_token_unix":1779713835.3266873,"t_done":282994.682767359,"t_done_unix":1779713859.6967106},{"request_id":"1253804:25:1662273:1020","session_id":"1253804","input_length":98187,"t_proxy_recv":283000.039851467,"t_decision_unix":1779713865.0537927,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":97792,"affinity_cache_ratio":0.9959770641734649,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":97792,"estimated_new_tokens":395,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":98187,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":83204,"ongoing_decode_tokens":83204,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1869,"cache_hit":97792,"new_prefill":395,"score_linear":-14588.0,"score_lmetric":395},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":98187,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":94603,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":98187,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":98187,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":98187,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":98187,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-14588.0,"chosen_score_lmetric":395,"t_first_token":283000.531824574,"t_first_token_unix":1779713865.5457687,"t_done":283001.153368965,"t_done_unix":1779713866.1673126},{"request_id":"1399948:47:1664702:1022","session_id":"1399948","input_length":40261,"t_proxy_recv":283008.642701525,"t_decision_unix":1779713873.656643,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":39936,"affinity_cache_ratio":0.9919276719405876,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":39936,"estimated_new_tokens":325,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":40261,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":181458,"ongoing_decode_tokens":181458,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1869,"cache_hit":0,"new_prefill":40261,"score_linear":181458.0,"score_lmetric":80522},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":40261,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":40261,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":40261,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":40261,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":39936,"new_prefill":325,"score_linear":-39936.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":40261,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-39936.0,"chosen_score_lmetric":0,"t_first_token":283008.874334001,"t_first_token_unix":1779713873.8882773,"t_done":283009.287506826,"t_done_unix":1779713874.3014503},{"request_id":"1294611:15:1657992:1017","session_id":"1294611","input_length":83204,"t_proxy_recv":282985.802644551,"t_decision_unix":1779713850.816586,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":82944,"affinity_cache_ratio":0.9968751502331619,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":82944,"estimated_new_tokens":260,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":83204,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1867,"cache_hit":82944,"new_prefill":260,"score_linear":-82944.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":12800,"new_prefill":70404,"score_linear":-12800.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":12800,"new_prefill":70404,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":83204,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":80406,"ongoing_decode_tokens":80406,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2355,"cache_hit":0,"new_prefill":83204,"score_linear":80406.0,"score_lmetric":83204},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":0,"new_prefill":83204,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":83204,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-82944.0,"chosen_score_lmetric":0,"t_first_token":282985.989243916,"t_first_token_unix":1779713851.0031872,"t_done":283012.392875853,"t_done_unix":1779713877.406819},{"request_id":"1356885:8:1665690:1023","session_id":"1356885","input_length":132537,"t_proxy_recv":283011.858800039,"t_decision_unix":1779713876.8727415,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":131584,"affinity_cache_ratio":0.9928095550676415,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":131584,"estimated_new_tokens":953,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":132537,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":181458,"ongoing_decode_tokens":181458,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1869,"cache_hit":0,"new_prefill":132537,"score_linear":181458.0,"score_lmetric":265074},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":131584,"new_prefill":953,"score_linear":-131584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":6656,"new_prefill":125881,"score_linear":-6656.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":132537,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":132537,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":132537,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":132537,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-131584.0,"chosen_score_lmetric":0,"t_first_token":283012.522184836,"t_first_token_unix":1779713877.5361285,"t_done":283014.31269796,"t_done_unix":1779713879.3266416},{"request_id":"1399948:48:1666394:1024","session_id":"1399948","input_length":40343,"t_proxy_recv":283014.271690884,"t_decision_unix":1779713879.2856324,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":39936,"affinity_cache_ratio":0.9899115088119377,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":39936,"estimated_new_tokens":407,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":40343,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":98254,"ongoing_decode_tokens":98254,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1870,"cache_hit":0,"new_prefill":40343,"score_linear":98254.0,"score_lmetric":40343},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":132537,"ongoing_decode_tokens":132537,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":40343,"score_linear":132537.0,"score_lmetric":40343},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":40343,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":40343,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":40343,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":39936,"new_prefill":407,"score_linear":-39936.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":40343,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-39936.0,"chosen_score_lmetric":0,"t_first_token":283014.356336333,"t_first_token_unix":1779713879.3702793,"t_done":283015.737677937,"t_done_unix":1779713880.751621},{"request_id":"1399948:49:1668825:1025","session_id":"1399948","input_length":40642,"t_proxy_recv":283022.83499345,"t_decision_unix":1779713887.8489347,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9952266128635402,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":194,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":40642,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":98254,"ongoing_decode_tokens":98254,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1870,"cache_hit":0,"new_prefill":40642,"score_linear":98254.0,"score_lmetric":40642},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":40642,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":40642,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":40642,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":40642,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2243,"cache_hit":40448,"new_prefill":194,"score_linear":-40448.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":40642,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":283022.965126113,"t_first_token_unix":1779713887.97907,"t_done":283024.034775272,"t_done_unix":1779713889.0487187},{"request_id":"1294611:15:1673197:1026","session_id":"1294611","input_length":85542,"t_proxy_recv":283038.05192112,"t_decision_unix":1779713903.0658624,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":84992,"affinity_cache_ratio":0.9935704098571462,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":84992,"estimated_new_tokens":550,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":85542,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":98254,"ongoing_decode_tokens":98254,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1870,"cache_hit":84992,"new_prefill":550,"score_linear":13262.0,"score_lmetric":550},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":12800,"new_prefill":72742,"score_linear":-12800.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":12800,"new_prefill":72742,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":85542,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":85542,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2243,"cache_hit":0,"new_prefill":85542,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":85542,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":13262.0,"chosen_score_lmetric":550,"t_first_token":283038.220990088,"t_first_token_unix":1779713903.2349336,"t_done":283040.784064809,"t_done_unix":1779713905.7980087},{"request_id":"1253804:26:1663639:1021","session_id":"1253804","input_length":98254,"t_proxy_recv":283004.874271131,"t_decision_unix":1779713869.888212,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":97792,"affinity_cache_ratio":0.9952979013577056,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":97792,"estimated_new_tokens":462,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":98254,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":83204,"ongoing_decode_tokens":83204,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1869,"cache_hit":97792,"new_prefill":462,"score_linear":-14588.0,"score_lmetric":462},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":0,"new_prefill":98254,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":94670,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":98254,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":98254,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":98254,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":98254,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-14588.0,"chosen_score_lmetric":462,"t_first_token":283005.050289752,"t_first_token_unix":1779713870.0642335,"t_done":283044.686747664,"t_done_unix":1779713909.7006912},{"request_id":"1294611:15:1676489:1027","session_id":"1294611","input_length":85722,"t_proxy_recv":283049.436666529,"t_decision_unix":1779713914.4506073,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":85504,"affinity_cache_ratio":0.9974568955460675,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":85504,"estimated_new_tokens":218,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":85722,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":85504,"new_prefill":218,"score_linear":-85504.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":12800,"new_prefill":72922,"score_linear":-12800.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":12800,"new_prefill":72922,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":85722,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":85722,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2243,"cache_hit":0,"new_prefill":85722,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":85722,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-85504.0,"chosen_score_lmetric":0,"t_first_token":283049.564876408,"t_first_token_unix":1779713914.57882,"t_done":283052.158899883,"t_done_unix":1779713917.172844},{"request_id":"1399948:50:1677482:1028","session_id":"1399948","input_length":42648,"t_proxy_recv":283052.690537437,"t_decision_unix":1779713917.7044783,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9484149315325455,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":2200,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2243,"cache_hit":40448,"new_prefill":2200,"score_linear":-40448.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":283053.282204159,"t_first_token_unix":1779713918.2961478,"t_done":283056.941275574,"t_done_unix":1779713921.9552193},{"request_id":"1294611:15:1679070:1029","session_id":"1294611","input_length":86602,"t_proxy_recv":283058.116914336,"t_decision_unix":1779713923.1308558,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":85504,"affinity_cache_ratio":0.987321308976698,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":85504,"estimated_new_tokens":1098,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":86602,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1872,"cache_hit":85504,"new_prefill":1098,"score_linear":-85504.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":12800,"new_prefill":73802,"score_linear":-12800.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":12800,"new_prefill":73802,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":86602,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":86602,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2248,"cache_hit":0,"new_prefill":86602,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":86602,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-85504.0,"chosen_score_lmetric":0,"t_first_token":283058.564773101,"t_first_token_unix":1779713923.578717,"t_done":283061.351061862,"t_done_unix":1779713926.3650053},{"request_id":"1253804:27:1683133:1031","session_id":"1253804","input_length":101743,"t_proxy_recv":283071.755026424,"t_decision_unix":1779713936.7689676,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101376,"affinity_cache_ratio":0.9963928722369106,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":101376,"estimated_new_tokens":367,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":101743,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":86925,"ongoing_decode_tokens":86925,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1874,"cache_hit":101376,"new_prefill":367,"score_linear":-14451.0,"score_lmetric":367},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":101743,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":98159,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":101743,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":101743,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2248,"cache_hit":0,"new_prefill":101743,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":101743,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-14451.0,"chosen_score_lmetric":367,"t_first_token":283071.969486021,"t_first_token_unix":1779713936.98343,"t_done":283072.449418744,"t_done_unix":1779713937.4633622},{"request_id":"1294611:15:1682649:1030","session_id":"1294611","input_length":86925,"t_proxy_recv":283070.026838379,"t_decision_unix":1779713935.0407796,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":86528,"affinity_cache_ratio":0.9954328444060973,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":86528,"estimated_new_tokens":397,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":86925,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1874,"cache_hit":86528,"new_prefill":397,"score_linear":-86528.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":12800,"new_prefill":74125,"score_linear":-12800.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":12800,"new_prefill":74125,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":86925,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":86925,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2248,"cache_hit":0,"new_prefill":86925,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":86925,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-86528.0,"chosen_score_lmetric":0,"t_first_token":283070.166272636,"t_first_token_unix":1779713935.1802168,"t_done":283074.472601239,"t_done_unix":1779713939.4865448},{"request_id":"1253804:28:1685257:1032","session_id":"1253804","input_length":101794,"t_proxy_recv":283078.751171146,"t_decision_unix":1779713943.7651122,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101376,"affinity_cache_ratio":0.9958936676031986,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":101376,"estimated_new_tokens":418,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":101794,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1875,"cache_hit":101376,"new_prefill":418,"score_linear":-101376.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":101794,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":98210,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":101794,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":101794,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2248,"cache_hit":0,"new_prefill":101794,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":101794,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-101376.0,"chosen_score_lmetric":0,"t_first_token":283078.902133735,"t_first_token_unix":1779713943.9160771,"t_done":283079.61917998,"t_done_unix":1779713944.6331234},{"request_id":"1399948:51:1686626:1033","session_id":"1399948","input_length":43407,"t_proxy_recv":283083.679566182,"t_decision_unix":1779713948.6935077,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":43008,"affinity_cache_ratio":0.9908079342041606,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":43008,"estimated_new_tokens":399,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1875,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2248,"cache_hit":43008,"new_prefill":399,"score_linear":-43008.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-43008.0,"chosen_score_lmetric":0,"t_first_token":283083.841052924,"t_first_token_unix":1779713948.8549964,"t_done":283085.282713143,"t_done_unix":1779713950.2966564},{"request_id":"1253804:29:1688398:1034","session_id":"1253804","input_length":102067,"t_proxy_recv":283089.733712938,"t_decision_unix":1779713954.7476542,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101376,"affinity_cache_ratio":0.9932299371981149,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":101376,"estimated_new_tokens":691,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":102067,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1875,"cache_hit":101376,"new_prefill":691,"score_linear":-101376.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":102067,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":98483,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":102067,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":102067,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":102067,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":102067,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-101376.0,"chosen_score_lmetric":0,"t_first_token":283090.050916351,"t_first_token_unix":1779713955.06486,"t_done":283091.06976351,"t_done_unix":1779713956.0837069},{"request_id":"1399948:52:1689144:1035","session_id":"1399948","input_length":43673,"t_proxy_recv":283092.314812265,"t_decision_unix":1779713957.3287535,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":43520,"affinity_cache_ratio":0.9964966913195796,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":43520,"estimated_new_tokens":153,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1876,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":43520,"new_prefill":153,"score_linear":-43520.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-43520.0,"chosen_score_lmetric":0,"t_first_token":283092.428389306,"t_first_token_unix":1779713957.4423327,"t_done":283093.164261524,"t_done_unix":1779713958.178205},{"request_id":"1356885:9:1689871:1036","session_id":"1356885","input_length":132823,"t_proxy_recv":283094.69326095,"t_decision_unix":1779713959.7072022,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":132608,"affinity_cache_ratio":0.9983813044427546,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":132608,"estimated_new_tokens":215,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":132823,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1876,"cache_hit":0,"new_prefill":132823,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":132608,"new_prefill":215,"score_linear":-132608.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":6656,"new_prefill":126167,"score_linear":-6656.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":132823,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":132823,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":132823,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":132823,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-132608.0,"chosen_score_lmetric":0,"t_first_token":283094.980384265,"t_first_token_unix":1779713959.9943278,"t_done":283096.673426847,"t_done_unix":1779713961.68737},{"request_id":"1253804:30:1690362:1037","session_id":"1253804","input_length":102214,"t_proxy_recv":283096.345049931,"t_decision_unix":1779713961.3589911,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101888,"affinity_cache_ratio":0.9968106130275696,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":101888,"estimated_new_tokens":326,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":102214,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1876,"cache_hit":101888,"new_prefill":326,"score_linear":-101888.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":132823,"ongoing_decode_tokens":132823,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":102214,"score_linear":132823.0,"score_lmetric":102214},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":98630,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":102214,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":102214,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":102214,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":102214,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-101888.0,"chosen_score_lmetric":0,"t_first_token":283096.54216697,"t_first_token_unix":1779713961.5561101,"t_done":283097.304746153,"t_done_unix":1779713962.3186896},{"request_id":"1253804:31:1691981:1038","session_id":"1253804","input_length":102322,"t_proxy_recv":283101.923318398,"t_decision_unix":1779713966.9372594,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101888,"affinity_cache_ratio":0.9957584879107132,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":101888,"estimated_new_tokens":434,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":102322,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1876,"cache_hit":101888,"new_prefill":434,"score_linear":-101888.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":102322,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":98738,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":102322,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":102322,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":102322,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":102322,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-101888.0,"chosen_score_lmetric":0,"t_first_token":283102.08324949,"t_first_token_unix":1779713967.0971935,"t_done":283102.464192851,"t_done_unix":1779713967.4781365},{"request_id":"1399948:53:1692115:1039","session_id":"1399948","input_length":43938,"t_proxy_recv":283102.436456246,"t_decision_unix":1779713967.4503973,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":43520,"affinity_cache_ratio":0.9904865947471437,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":43520,"estimated_new_tokens":418,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":43938,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102322,"ongoing_decode_tokens":102322,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1876,"cache_hit":0,"new_prefill":43938,"score_linear":102322.0,"score_lmetric":43938},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":43938,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":43938,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":43938,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":43938,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":43520,"new_prefill":418,"score_linear":-43520.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":43938,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-43520.0,"chosen_score_lmetric":0,"t_first_token":283102.593657839,"t_first_token_unix":1779713967.6076014,"t_done":283103.966534922,"t_done_unix":1779713968.9804783},{"request_id":"1253804:32:1693715:1040","session_id":"1253804","input_length":102509,"t_proxy_recv":283108.00739434,"t_decision_unix":1779713973.0213356,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101888,"affinity_cache_ratio":0.9939419953369948,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":101888,"estimated_new_tokens":621,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":102509,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1876,"cache_hit":101888,"new_prefill":621,"score_linear":-101888.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":102509,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":98925,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":102509,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":102509,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2250,"cache_hit":0,"new_prefill":102509,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":102509,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-101888.0,"chosen_score_lmetric":0,"t_first_token":283108.32229122,"t_first_token_unix":1779713973.3362348,"t_done":283109.102629885,"t_done_unix":1779713974.116573},{"request_id":"1399948:54:1694570:1041","session_id":"1399948","input_length":44182,"t_proxy_recv":283111.095542296,"t_decision_unix":1779713976.109483,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":44032,"affinity_cache_ratio":0.9966049522429948,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":44032,"estimated_new_tokens":150,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1877,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2250,"cache_hit":44032,"new_prefill":150,"score_linear":-44032.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-44032.0,"chosen_score_lmetric":0,"t_first_token":283111.206921092,"t_first_token_unix":1779713976.2208645,"t_done":283111.485056915,"t_done_unix":1779713976.499},{"request_id":"1253804:33:1695393:1042","session_id":"1253804","input_length":102632,"t_proxy_recv":283113.764808133,"t_decision_unix":1779713978.778749,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":102400,"affinity_cache_ratio":0.9977394964533479,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":102400,"estimated_new_tokens":232,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":102632,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1877,"cache_hit":102400,"new_prefill":232,"score_linear":-102400.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":102632,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":99048,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":102632,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":102632,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2250,"cache_hit":0,"new_prefill":102632,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":102632,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-102400.0,"chosen_score_lmetric":0,"t_first_token":283113.962072825,"t_first_token_unix":1779713978.9760163,"t_done":283114.463149706,"t_done_unix":1779713979.477093},{"request_id":"1253804:34:1697569:1043","session_id":"1253804","input_length":102792,"t_proxy_recv":283121.597515928,"t_decision_unix":1779713986.611457,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":102400,"affinity_cache_ratio":0.9961864736555374,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":102400,"estimated_new_tokens":392,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":102792,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1877,"cache_hit":102400,"new_prefill":392,"score_linear":-102400.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":102792,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":99208,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":102792,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":102792,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2250,"cache_hit":0,"new_prefill":102792,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":102792,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-102400.0,"chosen_score_lmetric":0,"t_first_token":283121.852949292,"t_first_token_unix":1779713986.866893,"t_done":283125.26077117,"t_done_unix":1779713990.2747147},{"request_id":"1253804:35:1701250:1044","session_id":"1253804","input_length":103349,"t_proxy_recv":283134.140975275,"t_decision_unix":1779713999.1549163,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":102912,"affinity_cache_ratio":0.9957716088205981,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":102912,"estimated_new_tokens":437,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":103349,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1878,"cache_hit":102912,"new_prefill":437,"score_linear":-102912.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":103349,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":99765,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":103349,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":103349,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2250,"cache_hit":0,"new_prefill":103349,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":103349,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-102912.0,"chosen_score_lmetric":0,"t_first_token":283134.467723876,"t_first_token_unix":1779713999.4816673,"t_done":283134.772215785,"t_done_unix":1779713999.7861593},{"request_id":"1253804:36:1702697:1045","session_id":"1253804","input_length":103400,"t_proxy_recv":283139.291552207,"t_decision_unix":1779714004.3054924,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":102912,"affinity_cache_ratio":0.9952804642166344,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":102912,"estimated_new_tokens":488,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":103400,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1878,"cache_hit":102912,"new_prefill":488,"score_linear":-102912.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":103400,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":99816,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":103400,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":103400,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2250,"cache_hit":0,"new_prefill":103400,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":103400,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-102912.0,"chosen_score_lmetric":0,"t_first_token":283139.439412953,"t_first_token_unix":1779714004.4533567,"t_done":283139.88739162,"t_done_unix":1779714004.9013352},{"request_id":"1399948:55:1704272:1047","session_id":"1399948","input_length":45284,"t_proxy_recv":283144.56041029,"t_decision_unix":1779714009.5743515,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":44032,"affinity_cache_ratio":0.9723522657009098,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":44032,"estimated_new_tokens":1252,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":45284,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103471,"ongoing_decode_tokens":103471,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":0,"new_prefill":45284,"score_linear":103471.0,"score_lmetric":45284},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":45284,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":45284,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":45284,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":45284,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2250,"cache_hit":44032,"new_prefill":1252,"score_linear":-44032.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":45284,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-44032.0,"chosen_score_lmetric":0,"t_first_token":283144.997758978,"t_first_token_unix":1779714010.0117025,"t_done":283146.675951917,"t_done_unix":1779714011.6898956},{"request_id":"1356885:10:1706624:1048","session_id":"1356885","input_length":133014,"t_proxy_recv":283152.888112678,"t_decision_unix":1779714017.9020536,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":132608,"affinity_cache_ratio":0.9969476897168719,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":132608,"estimated_new_tokens":406,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":133014,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103471,"ongoing_decode_tokens":103471,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":0,"new_prefill":133014,"score_linear":103471.0,"score_lmetric":133014},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":132608,"new_prefill":406,"score_linear":-132608.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":6656,"new_prefill":126358,"score_linear":-6656.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":133014,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":133014,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2252,"cache_hit":0,"new_prefill":133014,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":133014,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-132608.0,"chosen_score_lmetric":0,"t_first_token":283153.150434123,"t_first_token_unix":1779714018.1643775,"t_done":283154.621119065,"t_done_unix":1779714019.6350627},{"request_id":"1253804:37:1704071:1046","session_id":"1253804","input_length":103471,"t_proxy_recv":283143.763692411,"t_decision_unix":1779714008.7776332,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":103424,"affinity_cache_ratio":0.9995457664466372,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":103424,"estimated_new_tokens":47,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":103471,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":103424,"new_prefill":47,"score_linear":-103424.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":103471,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":99887,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":103471,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":103471,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2250,"cache_hit":0,"new_prefill":103471,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":103471,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-103424.0,"chosen_score_lmetric":0,"t_first_token":283143.912827995,"t_first_token_unix":1779714008.9267712,"t_done":283159.449640563,"t_done_unix":1779714024.4635844},{"request_id":"1399948:56:1708284:1049","session_id":"1399948","input_length":45571,"t_proxy_recv":283158.598241238,"t_decision_unix":1779714023.612182,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":45056,"affinity_cache_ratio":0.9886989532816923,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":45056,"estimated_new_tokens":515,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":45571,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103471,"ongoing_decode_tokens":103471,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":0,"new_prefill":45571,"score_linear":103471.0,"score_lmetric":45571},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":45571,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":45571,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":45571,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":45571,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2252,"cache_hit":45056,"new_prefill":515,"score_linear":-45056.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":45571,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-45056.0,"chosen_score_lmetric":0,"t_first_token":283158.717959789,"t_first_token_unix":1779714023.7319038,"t_done":283159.61656789,"t_done_unix":1779714024.630511},{"request_id":"1399948:57:1711464:1050","session_id":"1399948","input_length":45756,"t_proxy_recv":283169.386215412,"t_decision_unix":1779714034.4001563,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":45568,"affinity_cache_ratio":0.995891249235073,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":45568,"estimated_new_tokens":188,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2253,"cache_hit":45568,"new_prefill":188,"score_linear":-45568.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-45568.0,"chosen_score_lmetric":0,"t_first_token":283169.517897651,"t_first_token_unix":1779714034.531841,"t_done":283171.982978589,"t_done_unix":1779714036.9969218},{"request_id":"1253804:38:1712824:1051","session_id":"1253804","input_length":104953,"t_proxy_recv":283174.218007461,"t_decision_unix":1779714039.2319484,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":104448,"affinity_cache_ratio":0.9951883223919278,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":104448,"estimated_new_tokens":505,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":104953,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":104448,"new_prefill":505,"score_linear":-104448.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":104953,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":101369,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":104953,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":104953,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2254,"cache_hit":0,"new_prefill":104953,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":104953,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-104448.0,"chosen_score_lmetric":0,"t_first_token":283174.422442147,"t_first_token_unix":1779714039.4363859,"t_done":283174.734819973,"t_done_unix":1779714039.7487633},{"request_id":"1253804:39:1714767:1052","session_id":"1253804","input_length":105004,"t_proxy_recv":283180.802390585,"t_decision_unix":1779714045.8163319,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":104960,"affinity_cache_ratio":0.9995809683440631,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":104960,"estimated_new_tokens":44,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":105004,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":104960,"new_prefill":44,"score_linear":-104960.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":105004,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":101420,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":105004,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":105004,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2254,"cache_hit":0,"new_prefill":105004,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":105004,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-104960.0,"chosen_score_lmetric":0,"t_first_token":283180.964552254,"t_first_token_unix":1779714045.978496,"t_done":283181.527321763,"t_done_unix":1779714046.5412655},{"request_id":"1253804:40:1716790:1053","session_id":"1253804","input_length":105156,"t_proxy_recv":283187.51577606,"t_decision_unix":1779714052.5297172,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":104960,"affinity_cache_ratio":0.9981361025523984,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":104960,"estimated_new_tokens":196,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":105156,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":104960,"new_prefill":196,"score_linear":-104960.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":105156,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":101572,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":105156,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":105156,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2254,"cache_hit":0,"new_prefill":105156,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":105156,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-104960.0,"chosen_score_lmetric":0,"t_first_token":283187.774098022,"t_first_token_unix":1779714052.7880418,"t_done":283192.445160431,"t_done_unix":1779714057.4591038},{"request_id":"1305906:4:1718131:1054","session_id":"1305906","input_length":47861,"t_proxy_recv":283191.961011076,"t_decision_unix":1779714056.9749522,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":45056,"affinity_cache_ratio":0.9413927832682142,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":45056,"estimated_new_tokens":2805,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":47861,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":105156,"ongoing_decode_tokens":105156,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":0,"new_prefill":47861,"score_linear":105156.0,"score_lmetric":47861},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":47861,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":47861,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":16896,"new_prefill":30965,"score_linear":-16896.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":45056,"new_prefill":2805,"score_linear":-45056.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2254,"cache_hit":0,"new_prefill":47861,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":47861,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-45056.0,"chosen_score_lmetric":0,"t_first_token":283192.896259136,"t_first_token_unix":1779714057.9102027,"t_done":283194.980063802,"t_done_unix":1779714059.994007},{"request_id":"1305906:5:1514292:934","session_id":"1305906","input_length":49817,"t_proxy_recv":283194.991236048,"t_decision_unix":1779714060.0051775,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":48128,"affinity_cache_ratio":0.9660959110343859,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":48128,"estimated_new_tokens":1689,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":16896,"new_prefill":32921,"score_linear":-16896.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2367,"cache_hit":48128,"new_prefill":1689,"score_linear":-48128.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37173,"ongoing_decode_tokens":0,"pending_prefill_tokens":1845,"num_requests":1,"active_p_offloads":0,"cached_blocks":2254,"cache_hit":0,"new_prefill":49817,"score_linear":37173.0,"score_lmetric":51662},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-48128.0,"chosen_score_lmetric":0,"t_first_token":283195.653052275,"t_first_token_unix":1779714060.6669962,"t_done":283196.325618388,"t_done_unix":1779714061.3395617},{"request_id":"1340290:13:1718889:1055","session_id":"1340290","input_length":37173,"t_proxy_recv":283194.593813575,"t_decision_unix":1779714059.607755,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9503672020014526,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35328,"estimated_new_tokens":1845,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":47861,"ongoing_decode_tokens":47861,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":0,"new_prefill":37173,"score_linear":47861.0,"score_lmetric":37173},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2254,"cache_hit":35328,"new_prefill":1845,"score_linear":-35328.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-35328.0,"chosen_score_lmetric":0,"t_first_token":283200.259010611,"t_first_token_unix":1779714065.2729554,"t_done":283201.881347749,"t_done_unix":1779714066.8952909},{"request_id":"1253804:41:1721979:1057","session_id":"1253804","input_length":105638,"t_proxy_recv":283205.189475192,"t_decision_unix":1779714070.2034166,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105472,"affinity_cache_ratio":0.9984285957704614,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":105472,"estimated_new_tokens":166,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":105638,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":105472,"new_prefill":166,"score_linear":-105472.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":105638,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":102054,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":105638,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":105638,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37438,"ongoing_decode_tokens":37438,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2258,"cache_hit":0,"new_prefill":105638,"score_linear":37438.0,"score_lmetric":105638},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":105638,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105472.0,"chosen_score_lmetric":0,"t_first_token":283205.391767879,"t_first_token_unix":1779714070.4057112,"t_done":283206.042642114,"t_done_unix":1779714071.0565858},{"request_id":"1340290:14:1721731:1056","session_id":"1340290","input_length":37438,"t_proxy_recv":283204.315096379,"t_decision_unix":1779714069.3290374,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":37376,"affinity_cache_ratio":0.998343928628666,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":37376,"estimated_new_tokens":62,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2258,"cache_hit":37376,"new_prefill":62,"score_linear":-37376.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-37376.0,"chosen_score_lmetric":0,"t_first_token":283204.39046894,"t_first_token_unix":1779714069.4044123,"t_done":283206.162284854,"t_done_unix":1779714071.176228},{"request_id":"1253804:42:1724453:1058","session_id":"1253804","input_length":105715,"t_proxy_recv":283213.692355305,"t_decision_unix":1779714078.7062967,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105472,"affinity_cache_ratio":0.9977013668826562,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":105472,"estimated_new_tokens":243,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":105715,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":105472,"new_prefill":243,"score_linear":-105472.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":105715,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":102131,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":105715,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":105715,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2258,"cache_hit":0,"new_prefill":105715,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":105715,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105472.0,"chosen_score_lmetric":0,"t_first_token":283213.847093539,"t_first_token_unix":1779714078.8610368,"t_done":283214.62634162,"t_done_unix":1779714079.6402853},{"request_id":"1253804:43:1726322:1060","session_id":"1253804","input_length":105845,"t_proxy_recv":283219.764905283,"t_decision_unix":1779714084.7788465,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105472,"affinity_cache_ratio":0.9964759790259341,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":105472,"estimated_new_tokens":373,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":105845,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":105472,"new_prefill":373,"score_linear":-105472.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":105845,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":102261,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":105845,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":105845,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37724,"ongoing_decode_tokens":37724,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2258,"cache_hit":0,"new_prefill":105845,"score_linear":37724.0,"score_lmetric":105845},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":105845,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105472.0,"chosen_score_lmetric":0,"t_first_token":283219.974324981,"t_first_token_unix":1779714084.9882686,"t_done":283220.663437917,"t_done_unix":1779714085.6773813},{"request_id":"1399948:58:1726693:1061","session_id":"1399948","input_length":46886,"t_proxy_recv":283220.980898493,"t_decision_unix":1779714085.99484,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":46080,"affinity_cache_ratio":0.982809367401783,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":46080,"estimated_new_tokens":806,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37724,"ongoing_decode_tokens":37724,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2258,"cache_hit":46080,"new_prefill":806,"score_linear":-8356.0,"score_lmetric":806},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-8356.0,"chosen_score_lmetric":806,"t_first_token":283221.336134561,"t_first_token_unix":1779714086.3500779,"t_done":283221.526767132,"t_done_unix":1779714086.5407104},{"request_id":"1399948:59:1728305:1062","session_id":"1399948","input_length":46927,"t_proxy_recv":283226.275469376,"t_decision_unix":1779714091.2894106,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":46592,"affinity_cache_ratio":0.9928612525838004,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":46592,"estimated_new_tokens":335,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37724,"ongoing_decode_tokens":37724,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2259,"cache_hit":46592,"new_prefill":335,"score_linear":-8868.0,"score_lmetric":335},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-8868.0,"chosen_score_lmetric":335,"t_first_token":283226.368430757,"t_first_token_unix":1779714091.3823743,"t_done":283226.577699582,"t_done_unix":1779714091.5916429},{"request_id":"1253804:44:1728625:1063","session_id":"1253804","input_length":106070,"t_proxy_recv":283227.469867235,"t_decision_unix":1779714092.483808,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105472,"affinity_cache_ratio":0.9943622136325069,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":105472,"estimated_new_tokens":598,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":106070,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":105472,"new_prefill":598,"score_linear":-105472.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":106070,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":102486,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":106070,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":106070,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37724,"ongoing_decode_tokens":37724,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2259,"cache_hit":0,"new_prefill":106070,"score_linear":37724.0,"score_lmetric":106070},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":106070,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105472.0,"chosen_score_lmetric":0,"t_first_token":283227.797077937,"t_first_token_unix":1779714092.8110218,"t_done":283228.844053021,"t_done_unix":1779714093.8579974},{"request_id":"1340290:15:1725612:1059","session_id":"1340290","input_length":37724,"t_proxy_recv":283217.496571379,"t_decision_unix":1779714082.5105126,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":37376,"affinity_cache_ratio":0.9907751033824621,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":37376,"estimated_new_tokens":348,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2258,"cache_hit":37376,"new_prefill":348,"score_linear":-37376.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-37376.0,"chosen_score_lmetric":0,"t_first_token":283217.570889479,"t_first_token_unix":1779714082.5848324,"t_done":283235.571064047,"t_done_unix":1779714100.5850084},{"request_id":"1253804:45:1732408:1064","session_id":"1253804","input_length":106394,"t_proxy_recv":283240.435426188,"t_decision_unix":1779714105.449367,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105984,"affinity_cache_ratio":0.9961463992330395,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":105984,"estimated_new_tokens":410,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":106394,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1884,"cache_hit":105984,"new_prefill":410,"score_linear":-105984.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":106394,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":102810,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":106394,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":106394,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2262,"cache_hit":0,"new_prefill":106394,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":106394,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105984.0,"chosen_score_lmetric":0,"t_first_token":283240.762676037,"t_first_token_unix":1779714105.7766197,"t_done":283241.134117283,"t_done_unix":1779714106.148061},{"request_id":"1399948:60:1732843:1065","session_id":"1399948","input_length":47074,"t_proxy_recv":283242.028085938,"t_decision_unix":1779714107.0420272,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":46592,"affinity_cache_ratio":0.9897608021413095,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":46592,"estimated_new_tokens":482,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1884,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2262,"cache_hit":46592,"new_prefill":482,"score_linear":-46592.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-46592.0,"chosen_score_lmetric":0,"t_first_token":283242.151915571,"t_first_token_unix":1779714107.1658597,"t_done":283242.524595199,"t_done_unix":1779714107.5385387},{"request_id":"1253804:46:1733935:1066","session_id":"1253804","input_length":106443,"t_proxy_recv":283245.527362796,"t_decision_unix":1779714110.5413034,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105984,"affinity_cache_ratio":0.9956878329246639,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":105984,"estimated_new_tokens":459,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":106443,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1884,"cache_hit":105984,"new_prefill":459,"score_linear":-105984.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":106443,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":102859,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":106443,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":106443,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2263,"cache_hit":0,"new_prefill":106443,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":106443,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105984.0,"chosen_score_lmetric":0,"t_first_token":283245.665949737,"t_first_token_unix":1779714110.6798933,"t_done":283246.38104632,"t_done_unix":1779714111.39499},{"request_id":"1399948:61:1735933:1067","session_id":"1399948","input_length":47205,"t_proxy_recv":283252.224362052,"t_decision_unix":1779714117.238303,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":47104,"affinity_cache_ratio":0.9978603961444762,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":47104,"estimated_new_tokens":101,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1885,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2263,"cache_hit":47104,"new_prefill":101,"score_linear":-47104.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-47104.0,"chosen_score_lmetric":0,"t_first_token":283252.329432705,"t_first_token_unix":1779714117.343379,"t_done":283253.867328506,"t_done_unix":1779714118.8812723},{"request_id":"1253804:47:1736337:1068","session_id":"1253804","input_length":107137,"t_proxy_recv":283253.501894483,"t_decision_unix":1779714118.5158353,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":106496,"affinity_cache_ratio":0.994017006263009,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":106496,"estimated_new_tokens":641,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":107137,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1885,"cache_hit":106496,"new_prefill":641,"score_linear":-106496.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":107137,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":103553,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":107137,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":107137,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47205,"ongoing_decode_tokens":47205,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2263,"cache_hit":0,"new_prefill":107137,"score_linear":47205.0,"score_lmetric":107137},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":107137,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-106496.0,"chosen_score_lmetric":0,"t_first_token":283254.219829159,"t_first_token_unix":1779714119.2337732,"t_done":283254.633806583,"t_done_unix":1779714119.6477504},{"request_id":"1253804:48:1737694:1069","session_id":"1253804","input_length":108145,"t_proxy_recv":283257.942023481,"t_decision_unix":1779714122.9559643,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":107008,"affinity_cache_ratio":0.98948633778723,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":107008,"estimated_new_tokens":1137,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":108145,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1886,"cache_hit":107008,"new_prefill":1137,"score_linear":-107008.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":108145,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":104561,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":108145,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":108145,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2263,"cache_hit":0,"new_prefill":108145,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":108145,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-107008.0,"chosen_score_lmetric":0,"t_first_token":283258.867564026,"t_first_token_unix":1779714123.8815079,"t_done":283259.289971439,"t_done_unix":1779714124.3039148},{"request_id":"1253804:49:1739030:1070","session_id":"1253804","input_length":109454,"t_proxy_recv":283262.323251836,"t_decision_unix":1779714127.337193,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":108032,"affinity_cache_ratio":0.9870082409048551,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":108032,"estimated_new_tokens":1422,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":109454,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":108032,"new_prefill":1422,"score_linear":-108032.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":109454,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":105870,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":109454,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":109454,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2263,"cache_hit":0,"new_prefill":109454,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":109454,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-108032.0,"chosen_score_lmetric":0,"t_first_token":283263.458065607,"t_first_token_unix":1779714128.47201,"t_done":283263.886251025,"t_done_unix":1779714128.900195},{"request_id":"1253804:50:1741670:1071","session_id":"1253804","input_length":110854,"t_proxy_recv":283271.307251497,"t_decision_unix":1779714136.3211923,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":109056,"affinity_cache_ratio":0.9837804680029588,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":109056,"estimated_new_tokens":1798,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":110854,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1890,"cache_hit":109056,"new_prefill":1798,"score_linear":-109056.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":110854,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":107270,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":110854,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":110854,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2263,"cache_hit":0,"new_prefill":110854,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":110854,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-109056.0,"chosen_score_lmetric":0,"t_first_token":283272.46323715,"t_first_token_unix":1779714137.4771817,"t_done":283272.8958473,"t_done_unix":1779714137.9097908},{"request_id":"1253804:51:1744514:1072","session_id":"1253804","input_length":111661,"t_proxy_recv":283280.730356046,"t_decision_unix":1779714145.7442975,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":110592,"affinity_cache_ratio":0.9904263798461415,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":110592,"estimated_new_tokens":1069,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":111661,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1893,"cache_hit":110592,"new_prefill":1069,"score_linear":-110592.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":111661,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":108077,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":111661,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":111661,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2263,"cache_hit":0,"new_prefill":111661,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":111661,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-110592.0,"chosen_score_lmetric":0,"t_first_token":283281.485747885,"t_first_token_unix":1779714146.4996917,"t_done":283281.91725523,"t_done_unix":1779714146.9311984},{"request_id":"1340290:16:1745847:1073","session_id":"1340290","input_length":40200,"t_proxy_recv":283285.332371869,"t_decision_unix":1779714150.346313,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":39936,"affinity_cache_ratio":0.9934328358208955,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":39936,"estimated_new_tokens":264,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1895,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2263,"cache_hit":39936,"new_prefill":264,"score_linear":-39936.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-39936.0,"chosen_score_lmetric":0,"t_first_token":283285.411165457,"t_first_token_unix":1779714150.4251091,"t_done":283286.135471831,"t_done_unix":1779714151.149415},{"request_id":"1253804:52:1746191:1074","session_id":"1253804","input_length":111724,"t_proxy_recv":283286.3681704,"t_decision_unix":1779714151.3821115,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":111616,"affinity_cache_ratio":0.9990333321399162,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":111616,"estimated_new_tokens":108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":111724,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1895,"cache_hit":111616,"new_prefill":108,"score_linear":-111616.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":111724,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":108140,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":111724,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":111724,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2263,"cache_hit":0,"new_prefill":111724,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":111724,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-111616.0,"chosen_score_lmetric":0,"t_first_token":283286.527659513,"t_first_token_unix":1779714151.541603,"t_done":283287.308903473,"t_done_unix":1779714152.3228471},{"request_id":"1253804:53:1748377:1075","session_id":"1253804","input_length":111811,"t_proxy_recv":283293.707147703,"t_decision_unix":1779714158.7210891,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":111616,"affinity_cache_ratio":0.9982559855470392,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":111616,"estimated_new_tokens":195,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":111811,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1895,"cache_hit":111616,"new_prefill":195,"score_linear":-111616.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":111811,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":108227,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":111811,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":111811,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2263,"cache_hit":0,"new_prefill":111811,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":111811,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-111616.0,"chosen_score_lmetric":0,"t_first_token":283293.864619275,"t_first_token_unix":1779714158.878563,"t_done":283294.353767499,"t_done_unix":1779714159.367711},{"request_id":"1399948:62:1749028:1076","session_id":"1399948","input_length":48212,"t_proxy_recv":283295.866276459,"t_decision_unix":1779714160.8802176,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":47104,"affinity_cache_ratio":0.9770181697502697,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":47104,"estimated_new_tokens":1108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1895,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2263,"cache_hit":47104,"new_prefill":1108,"score_linear":-47104.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-47104.0,"chosen_score_lmetric":0,"t_first_token":283296.229698818,"t_first_token_unix":1779714161.2436423,"t_done":283296.784864935,"t_done_unix":1779714161.7988083},{"request_id":"1253804:54:1750056:1077","session_id":"1253804","input_length":112074,"t_proxy_recv":283299.300013728,"t_decision_unix":1779714164.3139548,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":111616,"affinity_cache_ratio":0.9959134143512323,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":111616,"estimated_new_tokens":458,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":112074,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1895,"cache_hit":111616,"new_prefill":458,"score_linear":-111616.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":112074,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":3584,"new_prefill":108490,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":112074,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":112074,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2265,"cache_hit":0,"new_prefill":112074,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":112074,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-111616.0,"chosen_score_lmetric":0,"t_first_token":283299.642688777,"t_first_token_unix":1779714164.6566324,"t_done":283305.599191075,"t_done_unix":1779714170.613135},{"request_id":"1399948:63:1753316:1079","session_id":"1399948","input_length":48636,"t_proxy_recv":283310.161218542,"t_decision_unix":1779714175.1751595,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":48128,"affinity_cache_ratio":0.9895550620939222,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":48128,"estimated_new_tokens":508,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":48636,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":48636,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":133552,"ongoing_decode_tokens":0,"pending_prefill_tokens":432,"num_requests":1,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":48636,"score_linear":133552.0,"score_lmetric":49068},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":48636,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":48636,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":48636,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2265,"cache_hit":48128,"new_prefill":508,"score_linear":-48128.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":48636,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-48128.0,"chosen_score_lmetric":0,"t_first_token":283310.422534345,"t_first_token_unix":1779714175.4364774,"t_done":283310.531308793,"t_done_unix":1779714175.5452518},{"request_id":"1399948:64:1754453:1080","session_id":"1399948","input_length":48693,"t_proxy_recv":283314.088134622,"t_decision_unix":1779714179.1020758,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":48640,"affinity_cache_ratio":0.9989115478610888,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":48640,"estimated_new_tokens":53,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":48693,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":48693,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":133552,"ongoing_decode_tokens":133552,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":0,"new_prefill":48693,"score_linear":133552.0,"score_lmetric":48693},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":48693,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":48693,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":48693,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2266,"cache_hit":48640,"new_prefill":53,"score_linear":-48640.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":48693,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-48640.0,"chosen_score_lmetric":0,"t_first_token":283314.19819974,"t_first_token_unix":1779714179.2121437,"t_done":283315.481245647,"t_done_unix":1779714180.4951894},{"request_id":"1356885:11:1753167:1078","session_id":"1356885","input_length":133552,"t_proxy_recv":283309.739134608,"t_decision_unix":1779714174.7530754,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":133120,"affinity_cache_ratio":0.9967653048999641,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":133120,"estimated_new_tokens":432,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":133552,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":133552,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":133120,"new_prefill":432,"score_linear":-133120.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":6656,"new_prefill":126896,"score_linear":-6656.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":133552,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":133552,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2265,"cache_hit":0,"new_prefill":133552,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":133552,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-133120.0,"chosen_score_lmetric":0,"t_first_token":283310.395861354,"t_first_token_unix":1779714175.4098048,"t_done":283327.79137936,"t_done_unix":1779714192.8053234},{"request_id":"1399948:65:1759200:1081","session_id":"1399948","input_length":49220,"t_proxy_recv":283330.570470613,"t_decision_unix":1779714195.5844116,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":48640,"affinity_cache_ratio":0.9882161722876879,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":48640,"estimated_new_tokens":580,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2266,"cache_hit":48640,"new_prefill":580,"score_linear":-48640.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-48640.0,"chosen_score_lmetric":0,"t_first_token":283330.853877805,"t_first_token_unix":1779714195.8678212,"t_done":283336.419378659,"t_done_unix":1779714201.4333217},{"request_id":"1277428:14:1767578:1082","session_id":"1277428","input_length":49716,"t_proxy_recv":283359.441756,"t_decision_unix":1779714224.4556973,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":49152,"affinity_cache_ratio":0.9886555636012552,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49152,"estimated_new_tokens":564,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":49716,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":4608,"new_prefill":45108,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":49716,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":49152,"new_prefill":564,"score_linear":-49152.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":49716,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":49716,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2268,"cache_hit":0,"new_prefill":49716,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":49716,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49152.0,"chosen_score_lmetric":0,"t_first_token":283359.619876301,"t_first_token_unix":1779714224.6338198,"t_done":283360.66690255,"t_done_unix":1779714225.680846},{"request_id":"1277428:14:1770237:1083","session_id":"1277428","input_length":49925,"t_proxy_recv":283368.77836971,"t_decision_unix":1779714233.7923107,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":49664,"affinity_cache_ratio":0.994772158237356,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":261,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":49925,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":4608,"new_prefill":45317,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":49925,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2129,"cache_hit":49664,"new_prefill":261,"score_linear":-49664.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":49925,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":49925,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2268,"cache_hit":0,"new_prefill":49925,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":49925,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":283368.890291322,"t_first_token_unix":1779714233.9042346,"t_done":283369.202887155,"t_done_unix":1779714234.216831},{"request_id":"1277428:15:1771467:1084","session_id":"1277428","input_length":51398,"t_proxy_recv":283372.849183002,"t_decision_unix":1779714237.8631241,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9662632787267987,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":1734,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":51398,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":4608,"new_prefill":46790,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":51398,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2129,"cache_hit":49664,"new_prefill":1734,"score_linear":-49664.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":51398,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":51398,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2268,"cache_hit":0,"new_prefill":51398,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":51398,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":283373.431455509,"t_first_token_unix":1779714238.4453995,"t_done":283376.942330889,"t_done_unix":1779714241.9562745},{"request_id":"1399948:66:1772336:1085","session_id":"1399948","input_length":51012,"t_proxy_recv":283375.798249145,"t_decision_unix":1779714240.8121889,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9735748451344781,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":1348,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":51012,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":51012,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":51012,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":51398,"ongoing_decode_tokens":51398,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2129,"cache_hit":0,"new_prefill":51012,"score_linear":51398.0,"score_lmetric":51012},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":51012,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":51012,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2268,"cache_hit":49664,"new_prefill":1348,"score_linear":-49664.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":51012,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":283376.278877057,"t_first_token_unix":1779714241.2928207,"t_done":283379.247418749,"t_done_unix":1779714244.261362},{"request_id":"1399948:67:1777074:1086","session_id":"1399948","input_length":52331,"t_proxy_recv":283391.870302109,"t_decision_unix":1779714256.884243,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":51200,"affinity_cache_ratio":0.9783875714203818,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":51200,"estimated_new_tokens":1131,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2133,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2271,"cache_hit":51200,"new_prefill":1131,"score_linear":-51200.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-51200.0,"chosen_score_lmetric":0,"t_first_token":283392.352675731,"t_first_token_unix":1779714257.3666196,"t_done":283396.680644745,"t_done_unix":1779714261.6945891},{"request_id":"1399948:68:1784799:1087","session_id":"1399948","input_length":53289,"t_proxy_recv":283418.365668162,"t_decision_unix":1779714283.3796089,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9896226238060387,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":52736,"estimated_new_tokens":553,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2133,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2274,"cache_hit":52736,"new_prefill":553,"score_linear":-52736.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":283418.653638884,"t_first_token_unix":1779714283.6675825,"t_done":283422.96909307,"t_done_unix":1779714287.9830368},{"request_id":"1277428:16:1787662:1088","session_id":"1277428","input_length":52754,"t_proxy_recv":283428.803846475,"t_decision_unix":1779714293.8177874,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":51712,"affinity_cache_ratio":0.9802479432839216,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":51712,"estimated_new_tokens":1042,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":52754,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":4608,"new_prefill":48146,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":52754,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2133,"cache_hit":51712,"new_prefill":1042,"score_linear":-51712.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":52754,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":0,"new_prefill":52754,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2276,"cache_hit":0,"new_prefill":52754,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":52754,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-51712.0,"chosen_score_lmetric":0,"t_first_token":283429.201836288,"t_first_token_unix":1779714294.215781,"t_done":283429.800051456,"t_done_unix":1779714294.8139951},{"request_id":"1305906:5:1788459:1089","session_id":"1305906","input_length":52461,"t_proxy_recv":283431.337504321,"t_decision_unix":1779714296.3514452,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":49664,"affinity_cache_ratio":0.946684203503555,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":2797,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":16896,"new_prefill":35565,"score_linear":-16896.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":49664,"new_prefill":2797,"score_linear":-49664.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2276,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":283432.326069605,"t_first_token_unix":1779714297.3400133,"t_done":283432.835034195,"t_done_unix":1779714297.8489776},{"request_id":"1399948:69:1790968:1090","session_id":"1399948","input_length":54141,"t_proxy_recv":283439.933109349,"t_decision_unix":1779714304.9470506,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":53760,"affinity_cache_ratio":0.9929628193051476,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":53760,"estimated_new_tokens":381,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":54141,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":54141,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":54141,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":0,"new_prefill":54141,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":54141,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":51255,"ongoing_decode_tokens":51255,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2375,"cache_hit":0,"new_prefill":54141,"score_linear":51255.0,"score_lmetric":54141},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2276,"cache_hit":53760,"new_prefill":381,"score_linear":-53760.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":54141,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53760.0,"chosen_score_lmetric":0,"t_first_token":283440.217246401,"t_first_token_unix":1779714305.2311897,"t_done":283444.110473516,"t_done_unix":1779714309.1244168},{"request_id":"1305906:6:1529600:945","session_id":"1305906","input_length":51255,"t_proxy_recv":283432.849208082,"t_decision_unix":1779714297.86315,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":51200,"affinity_cache_ratio":0.9989269339576626,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":51200,"estimated_new_tokens":55,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":16896,"new_prefill":34359,"score_linear":-16896.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2375,"cache_hit":51200,"new_prefill":55,"score_linear":-51200.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2276,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-51200.0,"chosen_score_lmetric":0,"t_first_token":283432.931738453,"t_first_token_unix":1779714297.9456818,"t_done":283447.973289054,"t_done_unix":1779714312.9872324},{"request_id":"1277428:17:1793546:1091","session_id":"1277428","input_length":53574,"t_proxy_recv":283448.936363096,"t_decision_unix":1779714313.9503043,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9843580841452944,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":52736,"estimated_new_tokens":838,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":53574,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":4608,"new_prefill":48966,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":53574,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2135,"cache_hit":52736,"new_prefill":838,"score_linear":-52736.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":53574,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2376,"cache_hit":0,"new_prefill":53574,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2277,"cache_hit":0,"new_prefill":53574,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":53574,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":283449.331165192,"t_first_token_unix":1779714314.3451087,"t_done":283450.649907677,"t_done_unix":1779714315.663851},{"request_id":"1399948:70:1795589:1092","session_id":"1399948","input_length":54654,"t_proxy_recv":283456.256107145,"t_decision_unix":1779714321.270048,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":54272,"affinity_cache_ratio":0.9930105756211806,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54272,"estimated_new_tokens":382,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2136,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2376,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2277,"cache_hit":54272,"new_prefill":382,"score_linear":-54272.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54272.0,"chosen_score_lmetric":0,"t_first_token":283456.345901882,"t_first_token_unix":1779714321.3598452,"t_done":283457.280505381,"t_done_unix":1779714322.2944486},{"request_id":"1277428:18:1796824:1093","session_id":"1277428","input_length":53805,"t_proxy_recv":283460.854009432,"t_decision_unix":1779714325.8679497,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":53248,"affinity_cache_ratio":0.9896478022488616,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":53248,"estimated_new_tokens":557,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":53805,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":4608,"new_prefill":49197,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":53805,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2136,"cache_hit":53248,"new_prefill":557,"score_linear":-53248.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":53805,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2376,"cache_hit":0,"new_prefill":53805,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2277,"cache_hit":0,"new_prefill":53805,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":53805,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53248.0,"chosen_score_lmetric":0,"t_first_token":283460.984728559,"t_first_token_unix":1779714325.9986722,"t_done":283461.358174743,"t_done_unix":1779714326.372118},{"request_id":"1399948:71:1800797:1094","session_id":"1399948","input_length":54920,"t_proxy_recv":283474.442086873,"t_decision_unix":1779714339.4560282,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":54272,"affinity_cache_ratio":0.9882010196649672,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54272,"estimated_new_tokens":648,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2137,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2376,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2277,"cache_hit":54272,"new_prefill":648,"score_linear":-54272.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54272.0,"chosen_score_lmetric":0,"t_first_token":283474.625906733,"t_first_token_unix":1779714339.6398504,"t_done":283476.702782748,"t_done_unix":1779714341.7167263},{"request_id":"1399948:72:1804699:1097","session_id":"1399948","input_length":55246,"t_proxy_recv":283487.893265184,"t_decision_unix":1779714352.9072063,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9916374036129313,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54784,"estimated_new_tokens":462,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":55246,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":55246,"score_linear":112634.0,"score_lmetric":55246},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":55246,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2137,"cache_hit":0,"new_prefill":55246,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":111736,"ongoing_decode_tokens":111736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":55246,"score_linear":111736.0,"score_lmetric":55246},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2376,"cache_hit":0,"new_prefill":55246,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2278,"cache_hit":54784,"new_prefill":462,"score_linear":-54784.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":55246,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":283488.012387473,"t_first_token_unix":1779714353.0263317,"t_done":283488.403780214,"t_done_unix":1779714353.4177237},{"request_id":"1399948:73:1805981:1098","session_id":"1399948","input_length":55559,"t_proxy_recv":283492.074472935,"t_decision_unix":1779714357.088414,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":55296,"affinity_cache_ratio":0.9952662934898037,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":55296,"estimated_new_tokens":263,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":55559,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":55559,"score_linear":112634.0,"score_lmetric":55559},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":55559,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2137,"cache_hit":0,"new_prefill":55559,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":111736,"ongoing_decode_tokens":111736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":55559,"score_linear":111736.0,"score_lmetric":55559},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2376,"cache_hit":0,"new_prefill":55559,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2279,"cache_hit":55296,"new_prefill":263,"score_linear":-55296.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":55559,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-55296.0,"chosen_score_lmetric":0,"t_first_token":283492.267570104,"t_first_token_unix":1779714357.2815137,"t_done":283493.559371996,"t_done_unix":1779714358.5733156},{"request_id":"1305906:6:1806419:1099","session_id":"1305906","input_length":52932,"t_proxy_recv":283493.584404219,"t_decision_unix":1779714358.5983453,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9962971359480087,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":52736,"estimated_new_tokens":196,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":52932,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":52932,"score_linear":112634.0,"score_lmetric":52932},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":52932,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2137,"cache_hit":0,"new_prefill":52932,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":111736,"ongoing_decode_tokens":111736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":16896,"new_prefill":36036,"score_linear":94840.0,"score_lmetric":36036},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2376,"cache_hit":52736,"new_prefill":196,"score_linear":-52736.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2279,"cache_hit":0,"new_prefill":52932,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":52932,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":283493.660706577,"t_first_token_unix":1779714358.6746504,"t_done":283494.373959807,"t_done_unix":1779714359.387903},{"request_id":"1305906:7:1572780:964","session_id":"1305906","input_length":54789,"t_proxy_recv":283494.385094178,"t_decision_unix":1779714359.399036,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9625289747942105,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":52736,"estimated_new_tokens":2053,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":54789,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":54789,"score_linear":112634.0,"score_lmetric":54789},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":54789,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2137,"cache_hit":0,"new_prefill":54789,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":111736,"ongoing_decode_tokens":111736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":16896,"new_prefill":37893,"score_linear":94840.0,"score_lmetric":37893},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2376,"cache_hit":52736,"new_prefill":2053,"score_linear":-52736.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2279,"cache_hit":0,"new_prefill":54789,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":54789,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":283495.105139875,"t_first_token_unix":1779714360.1190832,"t_done":283495.403406592,"t_done_unix":1779714360.4173496},{"request_id":"1399948:74:1808473:1100","session_id":"1399948","input_length":55791,"t_proxy_recv":283500.776868716,"t_decision_unix":1779714365.7908099,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":55296,"affinity_cache_ratio":0.9911276012260042,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":55296,"estimated_new_tokens":495,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":55791,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":55791,"score_linear":112634.0,"score_lmetric":55791},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":55791,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2137,"cache_hit":0,"new_prefill":55791,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":111736,"ongoing_decode_tokens":111736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":55791,"score_linear":111736.0,"score_lmetric":55791},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2380,"cache_hit":0,"new_prefill":55791,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2279,"cache_hit":55296,"new_prefill":495,"score_linear":-55296.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":55791,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-55296.0,"chosen_score_lmetric":0,"t_first_token":283500.911070491,"t_first_token_unix":1779714365.925014,"t_done":283502.789536595,"t_done_unix":1779714367.8034801},{"request_id":"1270606:6:1804345:1096","session_id":"1270606","input_length":111736,"t_proxy_recv":283486.735351304,"t_decision_unix":1779714351.7492924,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":111616,"affinity_cache_ratio":0.9989260399513138,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":111616,"estimated_new_tokens":120,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":111736,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":111736,"score_linear":112634.0,"score_lmetric":111736},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":111736,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2137,"cache_hit":0,"new_prefill":111736,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":111616,"new_prefill":120,"score_linear":-111616.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2376,"cache_hit":0,"new_prefill":111736,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2278,"cache_hit":0,"new_prefill":111736,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":110200,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-111616.0,"chosen_score_lmetric":0,"t_first_token":283487.026378866,"t_first_token_unix":1779714352.0403228,"t_done":283529.684878663,"t_done_unix":1779714394.6988225},{"request_id":"1399948:75:1817312:1101","session_id":"1399948","input_length":57308,"t_proxy_recv":283529.313425765,"t_decision_unix":1779714394.3273666,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":55808,"affinity_cache_ratio":0.9738256438891604,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":55808,"estimated_new_tokens":1500,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":57308,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":57308,"score_linear":112634.0,"score_lmetric":57308},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":57308,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2137,"cache_hit":0,"new_prefill":57308,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":111736,"ongoing_decode_tokens":111736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":57308,"score_linear":111736.0,"score_lmetric":57308},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2380,"cache_hit":0,"new_prefill":57308,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2280,"cache_hit":55808,"new_prefill":1500,"score_linear":-55808.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":57308,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-55808.0,"chosen_score_lmetric":0,"t_first_token":283529.950872551,"t_first_token_unix":1779714394.9648166,"t_done":283530.784283984,"t_done_unix":1779714395.7982273},{"request_id":"1399948:76:1821495:1102","session_id":"1399948","input_length":57659,"t_proxy_recv":283542.970546673,"t_decision_unix":1779714407.9844875,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":57344,"affinity_cache_ratio":0.9945368459390554,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57344,"estimated_new_tokens":315,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":57659,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":57659,"score_linear":112634.0,"score_lmetric":57659},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":57659,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2137,"cache_hit":0,"new_prefill":57659,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2224,"cache_hit":0,"new_prefill":57659,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2380,"cache_hit":0,"new_prefill":57659,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2283,"cache_hit":57344,"new_prefill":315,"score_linear":-57344.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":57659,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57344.0,"chosen_score_lmetric":0,"t_first_token":283543.171328878,"t_first_token_unix":1779714408.1852727,"t_done":283543.82896485,"t_done_unix":1779714408.8429081},{"request_id":"1270606:6:1822681:1103","session_id":"1270606","input_length":112180,"t_proxy_recv":283546.811207978,"t_decision_unix":1779714411.8251488,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":112128,"affinity_cache_ratio":0.9995364592619005,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":112128,"estimated_new_tokens":52,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":112180,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":112180,"score_linear":112634.0,"score_lmetric":112180},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":112180,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2137,"cache_hit":0,"new_prefill":112180,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2224,"cache_hit":112128,"new_prefill":52,"score_linear":-112128.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2380,"cache_hit":0,"new_prefill":112180,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2283,"cache_hit":0,"new_prefill":112180,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":110644,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-112128.0,"chosen_score_lmetric":0,"t_first_token":283546.958329735,"t_first_token_unix":1779714411.9722733,"t_done":283547.846505227,"t_done_unix":1779714412.8604493},{"request_id":"1270606:7:1825806:1104","session_id":"1270606","input_length":112645,"t_proxy_recv":283557.426720902,"t_decision_unix":1779714422.440662,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":112128,"affinity_cache_ratio":0.9954103599804697,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":112128,"estimated_new_tokens":517,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":112645,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":112645,"score_linear":112634.0,"score_lmetric":112645},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":112645,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2137,"cache_hit":0,"new_prefill":112645,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2224,"cache_hit":112128,"new_prefill":517,"score_linear":-112128.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2380,"cache_hit":0,"new_prefill":112645,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2283,"cache_hit":0,"new_prefill":112645,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":111109,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-112128.0,"chosen_score_lmetric":0,"t_first_token":283557.989523833,"t_first_token_unix":1779714423.0034676,"t_done":283562.056739376,"t_done_unix":1779714427.0706832},{"request_id":"1277428:19:1834491:1105","session_id":"1277428","input_length":58474,"t_proxy_recv":283586.698047335,"t_decision_unix":1779714451.7119882,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":53760,"affinity_cache_ratio":0.9193829736293053,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":53760,"estimated_new_tokens":4714,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":58474,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":4608,"new_prefill":53866,"score_linear":108026.0,"score_lmetric":53866},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":58474,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2137,"cache_hit":53760,"new_prefill":4714,"score_linear":-53760.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2225,"cache_hit":0,"new_prefill":58474,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2380,"cache_hit":0,"new_prefill":58474,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2283,"cache_hit":0,"new_prefill":58474,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":58474,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53760.0,"chosen_score_lmetric":0,"t_first_token":283588.435517179,"t_first_token_unix":1779714453.449461,"t_done":283588.767356257,"t_done_unix":1779714453.7812998},{"request_id":"1270606:8:1837578:1106","session_id":"1270606","input_length":113482,"t_proxy_recv":283597.529352021,"t_decision_unix":1779714462.5432935,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":112640,"affinity_cache_ratio":0.9925803211081934,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":112640,"estimated_new_tokens":842,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":113482,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":113482,"score_linear":112634.0,"score_lmetric":113482},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":113482,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":113482,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2225,"cache_hit":112640,"new_prefill":842,"score_linear":-112640.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2380,"cache_hit":0,"new_prefill":113482,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2283,"cache_hit":0,"new_prefill":113482,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":111946,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-112640.0,"chosen_score_lmetric":0,"t_first_token":283598.092963188,"t_first_token_unix":1779714463.1069071,"t_done":283598.525029663,"t_done_unix":1779714463.5389738},{"request_id":"1253804:55:1803044:1095","session_id":"1253804","input_length":112634,"t_proxy_recv":283482.320947351,"t_decision_unix":1779714347.3348885,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":112128,"affinity_cache_ratio":0.9955075732016976,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":112128,"estimated_new_tokens":506,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":112634,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":112128,"new_prefill":506,"score_linear":-112128.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":112634,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2137,"cache_hit":3584,"new_prefill":109050,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":112634,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2376,"cache_hit":0,"new_prefill":112634,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2278,"cache_hit":0,"new_prefill":112634,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":112634,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-112128.0,"chosen_score_lmetric":0,"t_first_token":283482.495391978,"t_first_token_unix":1779714347.5093355,"t_done":283607.468098831,"t_done_unix":1779714472.4820454},{"request_id":"1270606:9:1847569:1107","session_id":"1270606","input_length":116467,"t_proxy_recv":283631.43373519,"t_decision_unix":1779714496.4476764,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":113152,"affinity_cache_ratio":0.9715370018975332,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":113152,"estimated_new_tokens":3315,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2226,"cache_hit":113152,"new_prefill":3315,"score_linear":-113152.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2380,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2283,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":114931,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-113152.0,"chosen_score_lmetric":0,"t_first_token":283633.682821437,"t_first_token_unix":1779714498.6967652,"t_done":283635.219827112,"t_done_unix":1779714500.233771},{"request_id":"1270606:10:1851128:1108","session_id":"1270606","input_length":116659,"t_proxy_recv":283643.516407458,"t_decision_unix":1779714508.5303485,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":116224,"affinity_cache_ratio":0.9962711835349181,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":116224,"estimated_new_tokens":435,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2232,"cache_hit":116224,"new_prefill":435,"score_linear":-116224.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2380,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2283,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":115123,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-116224.0,"chosen_score_lmetric":0,"t_first_token":283643.766547982,"t_first_token_unix":1779714508.7804923,"t_done":283644.522722475,"t_done_unix":1779714509.5366666},{"request_id":"1305906:7:1851943:1109","session_id":"1305906","input_length":58042,"t_proxy_recv":283646.053412908,"t_decision_unix":1779714511.067354,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9438682333482651,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54784,"estimated_new_tokens":3258,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2232,"cache_hit":16896,"new_prefill":41146,"score_linear":-16896.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2380,"cache_hit":54784,"new_prefill":3258,"score_linear":-54784.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2283,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":283647.359932285,"t_first_token_unix":1779714512.3738759,"t_done":283649.612940658,"t_done_unix":1779714514.6268847},{"request_id":"1305906:8:1866131:1110","session_id":"1305906","input_length":58687,"t_proxy_recv":283694.524353596,"t_decision_unix":1779714559.5382946,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":57856,"affinity_cache_ratio":0.9858401349532264,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57856,"estimated_new_tokens":831,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2232,"cache_hit":16896,"new_prefill":41791,"score_linear":-16896.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2386,"cache_hit":57856,"new_prefill":831,"score_linear":-57856.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2283,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57856.0,"chosen_score_lmetric":0,"t_first_token":283694.846260676,"t_first_token_unix":1779714559.8602054,"t_done":283696.727164253,"t_done_unix":1779714561.7411082},{"request_id":"1270606:11:1868274:1111","session_id":"1270606","input_length":117828,"t_proxy_recv":283701.806130637,"t_decision_unix":1779714566.8200715,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":116224,"affinity_cache_ratio":0.9863869368910615,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":116224,"estimated_new_tokens":1604,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2232,"cache_hit":116224,"new_prefill":1604,"score_linear":-116224.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2283,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":116292,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-116224.0,"chosen_score_lmetric":0,"t_first_token":283702.817109873,"t_first_token_unix":1779714567.831055,"t_done":283704.557657939,"t_done_unix":1779714569.5716014},{"request_id":"1270606:11:1870810:1113","session_id":"1270606","input_length":118012,"t_proxy_recv":283710.303980487,"t_decision_unix":1779714575.3179216,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":117760,"affinity_cache_ratio":0.9978646239365488,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":117760,"estimated_new_tokens":252,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":118012,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":118012,"score_linear":112926.0,"score_lmetric":118012},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":118012,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":118012,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2235,"cache_hit":117760,"new_prefill":252,"score_linear":-117760.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":118012,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2283,"cache_hit":0,"new_prefill":118012,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":116476,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-117760.0,"chosen_score_lmetric":0,"t_first_token":283710.474970778,"t_first_token_unix":1779714575.4889143,"t_done":283710.984722175,"t_done_unix":1779714575.9986658},{"request_id":"1356885:12:1874205:1114","session_id":"1356885","input_length":135435,"t_proxy_recv":283721.761291045,"t_decision_unix":1779714586.7752323,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":134656,"affinity_cache_ratio":0.9942481633255805,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":134656,"estimated_new_tokens":779,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":135435,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":135435,"score_linear":112926.0,"score_lmetric":135435},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":134656,"new_prefill":779,"score_linear":-134656.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":6656,"new_prefill":128779,"score_linear":-6656.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2235,"cache_hit":0,"new_prefill":135435,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":135435,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2283,"cache_hit":0,"new_prefill":135435,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":135435,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-134656.0,"chosen_score_lmetric":0,"t_first_token":283722.441140243,"t_first_token_unix":1779714587.4550843,"t_done":283724.992454517,"t_done_unix":1779714590.006398},{"request_id":"1340290:17:1875101:1115","session_id":"1340290","input_length":40328,"t_proxy_recv":283724.692803884,"t_decision_unix":1779714589.7067447,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":39936,"affinity_cache_ratio":0.9902797064074589,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":39936,"estimated_new_tokens":392,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":40328,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":40328,"score_linear":112926.0,"score_lmetric":40328},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":135435,"ongoing_decode_tokens":135435,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":40328,"score_linear":135435.0,"score_lmetric":40328},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":40328,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2235,"cache_hit":0,"new_prefill":40328,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":40328,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2283,"cache_hit":39936,"new_prefill":392,"score_linear":-39936.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":40328,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-39936.0,"chosen_score_lmetric":0,"t_first_token":283724.776624824,"t_first_token_unix":1779714589.7905684,"t_done":283727.892701708,"t_done_unix":1779714592.906645},{"request_id":"1270606:12:1876851:1116","session_id":"1270606","input_length":118600,"t_proxy_recv":283730.658504452,"t_decision_unix":1779714595.6724453,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":117760,"affinity_cache_ratio":0.9929173693086003,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":117760,"estimated_new_tokens":840,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":118600,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":118600,"score_linear":112926.0,"score_lmetric":118600},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":118600,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":118600,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2235,"cache_hit":117760,"new_prefill":840,"score_linear":-117760.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":118600,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2284,"cache_hit":0,"new_prefill":118600,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":117064,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-117760.0,"chosen_score_lmetric":0,"t_first_token":283731.244095961,"t_first_token_unix":1779714596.2580395,"t_done":283733.413176137,"t_done_unix":1779714598.4271197},{"request_id":"1340290:18:1878955:1117","session_id":"1340290","input_length":40583,"t_proxy_recv":283738.36832921,"t_decision_unix":1779714603.38227,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9966734839711209,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":135,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":40583,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":40583,"score_linear":112926.0,"score_lmetric":40583},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":40583,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":40583,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2237,"cache_hit":0,"new_prefill":40583,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":40583,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2284,"cache_hit":40448,"new_prefill":135,"score_linear":-40448.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":40583,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":283738.431340629,"t_first_token_unix":1779714603.4452844,"t_done":283738.643413607,"t_done_unix":1779714603.657357},{"request_id":"1340290:19:1880565:1118","session_id":"1340290","input_length":40676,"t_proxy_recv":283743.640238287,"t_decision_unix":1779714608.6541798,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9943947290785722,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":228,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":40676,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":40676,"score_linear":112926.0,"score_lmetric":40676},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":40676,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":40676,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2237,"cache_hit":0,"new_prefill":40676,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":40676,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2284,"cache_hit":40448,"new_prefill":228,"score_linear":-40448.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":40676,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":283743.743937852,"t_first_token_unix":1779714608.7578814,"t_done":283744.301942606,"t_done_unix":1779714609.315886},{"request_id":"1340290:20:1882251:1120","session_id":"1340290","input_length":40883,"t_proxy_recv":283749.449567198,"t_decision_unix":1779714614.4635081,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9893598806349827,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":435,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":40883,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":40883,"score_linear":112926.0,"score_lmetric":40883},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":40883,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":40883,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":119030,"ongoing_decode_tokens":119030,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2237,"cache_hit":0,"new_prefill":40883,"score_linear":119030.0,"score_lmetric":40883},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":40883,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2284,"cache_hit":40448,"new_prefill":435,"score_linear":-40448.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":40883,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":283749.560231228,"t_first_token_unix":1779714614.5741746,"t_done":283749.885933663,"t_done_unix":1779714614.8998768},{"request_id":"1270606:12:1881189:1119","session_id":"1270606","input_length":119030,"t_proxy_recv":283746.053872029,"t_decision_unix":1779714611.0678132,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":118784,"affinity_cache_ratio":0.9979332941275308,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":118784,"estimated_new_tokens":246,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":119030,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":119030,"score_linear":112926.0,"score_lmetric":119030},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":119030,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":119030,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2237,"cache_hit":118784,"new_prefill":246,"score_linear":-118784.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":119030,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2284,"cache_hit":0,"new_prefill":119030,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":117494,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-118784.0,"chosen_score_lmetric":0,"t_first_token":283746.420926778,"t_first_token_unix":1779714611.4348702,"t_done":283750.458279742,"t_done_unix":1779714615.4722233},{"request_id":"1340290:21:1883855:1121","session_id":"1340290","input_length":41002,"t_proxy_recv":283754.91910214,"t_decision_unix":1779714619.9330432,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":40448,"affinity_cache_ratio":0.986488463977367,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":554,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":41002,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":41002,"score_linear":112926.0,"score_lmetric":41002},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":41002,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":41002,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":0,"new_prefill":41002,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":41002,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2284,"cache_hit":40448,"new_prefill":554,"score_linear":-40448.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":41002,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":283755.025278504,"t_first_token_unix":1779714620.0392227,"t_done":283755.278198996,"t_done_unix":1779714620.2921424},{"request_id":"1340290:22:1884847:1122","session_id":"1340290","input_length":41909,"t_proxy_recv":283758.391059696,"t_decision_unix":1779714623.4050007,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":40960,"affinity_cache_ratio":0.9773556992531437,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40960,"estimated_new_tokens":949,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":41909,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":41909,"score_linear":112926.0,"score_lmetric":41909},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":41909,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":41909,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":0,"new_prefill":41909,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":41909,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2285,"cache_hit":40960,"new_prefill":949,"score_linear":-40960.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":41909,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40960.0,"chosen_score_lmetric":0,"t_first_token":283758.718788159,"t_first_token_unix":1779714623.7327318,"t_done":283759.059233549,"t_done_unix":1779714624.0731769},{"request_id":"1253804:55:1869941:1112","session_id":"1253804","input_length":112926,"t_proxy_recv":283707.431566145,"t_decision_unix":1779714572.445507,"policy":"unified","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":112640,"affinity_cache_ratio":0.9974673680109098,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":112640,"estimated_new_tokens":286,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":112926,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":112640,"new_prefill":286,"score_linear":-112640.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":112926,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":3584,"new_prefill":109342,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2235,"cache_hit":0,"new_prefill":112926,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":112926,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2283,"cache_hit":0,"new_prefill":112926,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":112926,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-112640.0,"chosen_score_lmetric":0,"t_first_token":283707.580654966,"t_first_token_unix":1779714572.5945983,"t_done":283760.643481484,"t_done_unix":1779714625.6574538},{"request_id":"1340290:23:1886013:1124","session_id":"1340290","input_length":42086,"t_proxy_recv":283762.422075524,"t_decision_unix":1779714627.4360168,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":41472,"affinity_cache_ratio":0.9854108254526446,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":41472,"estimated_new_tokens":614,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":42086,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":42086,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":42086,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":63845,"ongoing_decode_tokens":0,"pending_prefill_tokens":5477,"num_requests":1,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":42086,"score_linear":63845.0,"score_lmetric":47563},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":0,"new_prefill":42086,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":42086,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2286,"cache_hit":41472,"new_prefill":614,"score_linear":-41472.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":42086,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-41472.0,"chosen_score_lmetric":0,"t_first_token":283762.533921149,"t_first_token_unix":1779714627.5478647,"t_done":283762.828856233,"t_done_unix":1779714627.8428},{"request_id":"1270606:12:1886765:1125","session_id":"1270606","input_length":119440,"t_proxy_recv":283765.166156406,"t_decision_unix":1779714630.1800973,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":119296,"affinity_cache_ratio":0.9987943737441393,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":119296,"estimated_new_tokens":144,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":119440,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":119440,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":119440,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":63845,"ongoing_decode_tokens":63845,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":119440,"score_linear":63845.0,"score_lmetric":119440},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":119296,"new_prefill":144,"score_linear":-119296.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":119440,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2287,"cache_hit":0,"new_prefill":119440,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":117904,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-119296.0,"chosen_score_lmetric":0,"t_first_token":283765.404687748,"t_first_token_unix":1779714630.4186318,"t_done":283766.457140232,"t_done_unix":1779714631.4710839},{"request_id":"1340290:24:1887125:1126","session_id":"1340290","input_length":42199,"t_proxy_recv":283766.198155868,"t_decision_unix":1779714631.2120972,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":41984,"affinity_cache_ratio":0.9949050925377378,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":41984,"estimated_new_tokens":215,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":42199,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":42199,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":42199,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":63845,"ongoing_decode_tokens":63845,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":42199,"score_linear":63845.0,"score_lmetric":42199},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":119440,"ongoing_decode_tokens":119440,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":0,"new_prefill":42199,"score_linear":119440.0,"score_lmetric":42199},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":42199,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2287,"cache_hit":41984,"new_prefill":215,"score_linear":-41984.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":42199,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-41984.0,"chosen_score_lmetric":0,"t_first_token":283766.305834629,"t_first_token_unix":1779714631.3197782,"t_done":283766.474152244,"t_done_unix":1779714631.4880953},{"request_id":"1340290:25:1888447:1127","session_id":"1340290","input_length":42444,"t_proxy_recv":283770.590905973,"t_decision_unix":1779714635.6048474,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":41984,"affinity_cache_ratio":0.9891621901800018,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":41984,"estimated_new_tokens":460,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":42444,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":42444,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":42444,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":63845,"ongoing_decode_tokens":63845,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":42444,"score_linear":63845.0,"score_lmetric":42444},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":0,"new_prefill":42444,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":42444,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2287,"cache_hit":41984,"new_prefill":460,"score_linear":-41984.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":42444,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-41984.0,"chosen_score_lmetric":0,"t_first_token":283770.743589509,"t_first_token_unix":1779714635.7575328,"t_done":283771.047023195,"t_done_unix":1779714636.0609665},{"request_id":"1277428:20:1885419:1123","session_id":"1277428","input_length":63845,"t_proxy_recv":283760.30799173,"t_decision_unix":1779714625.321933,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":58368,"affinity_cache_ratio":0.9142141123032343,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58368,"estimated_new_tokens":5477,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":63845,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":4608,"new_prefill":59237,"score_linear":108318.0,"score_lmetric":59237},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":63845,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":58368,"new_prefill":5477,"score_linear":-58368.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":0,"new_prefill":63845,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":63845,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2286,"cache_hit":0,"new_prefill":63845,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":63845,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58368.0,"chosen_score_lmetric":0,"t_first_token":283762.512890656,"t_first_token_unix":1779714627.5268347,"t_done":283772.670970526,"t_done_unix":1779714637.684914},{"request_id":"1340290:26:1889402:1128","session_id":"1340290","input_length":48504,"t_proxy_recv":283773.911201833,"t_decision_unix":1779714638.925143,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":41984,"affinity_cache_ratio":0.8655780966518225,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":41984,"estimated_new_tokens":6520,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2287,"cache_hit":41984,"new_prefill":6520,"score_linear":-41984.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-41984.0,"chosen_score_lmetric":0,"t_first_token":283775.814714679,"t_first_token_unix":1779714640.828658,"t_done":283776.021801926,"t_done_unix":1779714641.0357466},{"request_id":"1270606:13:1890252:1129","session_id":"1270606","input_length":119686,"t_proxy_recv":283776.894985945,"t_decision_unix":1779714641.908927,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":119296,"affinity_cache_ratio":0.9967414735223836,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":119296,"estimated_new_tokens":390,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":119296,"new_prefill":390,"score_linear":-119296.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2299,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":118150,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-119296.0,"chosen_score_lmetric":0,"t_first_token":283777.25856024,"t_first_token_unix":1779714642.272505,"t_done":283778.183248683,"t_done_unix":1779714643.1971922},{"request_id":"1340290:27:1892591:1131","session_id":"1340290","input_length":49361,"t_proxy_recv":283784.497997845,"t_decision_unix":1779714649.5119393,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":48128,"affinity_cache_ratio":0.9750207653815766,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":48128,"estimated_new_tokens":1233,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":49361,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":49361,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":49361,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":49361,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":119828,"ongoing_decode_tokens":119828,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":0,"new_prefill":49361,"score_linear":119828.0,"score_lmetric":49361},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":49361,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2299,"cache_hit":48128,"new_prefill":1233,"score_linear":-48128.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":49361,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-48128.0,"chosen_score_lmetric":0,"t_first_token":283784.870765838,"t_first_token_unix":1779714649.8847094,"t_done":283785.195157848,"t_done_unix":1779714650.2091012},{"request_id":"1270606:14:1892412:1130","session_id":"1270606","input_length":119828,"t_proxy_recv":283783.840638689,"t_decision_unix":1779714648.8545797,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":119296,"affinity_cache_ratio":0.9955603031011115,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":119296,"estimated_new_tokens":532,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":119296,"new_prefill":532,"score_linear":-119296.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2299,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":118292,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-119296.0,"chosen_score_lmetric":0,"t_first_token":283784.097819365,"t_first_token_unix":1779714649.111763,"t_done":283786.068617379,"t_done_unix":1779714651.082561},{"request_id":"1340290:28:1893837:1132","session_id":"1340290","input_length":49454,"t_proxy_recv":283788.806904775,"t_decision_unix":1779714653.820846,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":49152,"affinity_cache_ratio":0.9938933149997978,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49152,"estimated_new_tokens":302,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2301,"cache_hit":49152,"new_prefill":302,"score_linear":-49152.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49152.0,"chosen_score_lmetric":0,"t_first_token":283788.921925919,"t_first_token_unix":1779714653.9358695,"t_done":283789.171669618,"t_done_unix":1779714654.1856132},{"request_id":"1305906:9:1893880:1133","session_id":"1305906","input_length":59265,"t_proxy_recv":283788.971504178,"t_decision_unix":1779714653.985446,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9935037543237999,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":385,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":16896,"new_prefill":42369,"score_linear":-16896.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":58880,"new_prefill":385,"score_linear":-58880.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":49454,"ongoing_decode_tokens":49454,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2301,"cache_hit":0,"new_prefill":59265,"score_linear":49454.0,"score_lmetric":59265},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58880.0,"chosen_score_lmetric":0,"t_first_token":283789.306113422,"t_first_token_unix":1779714654.320057,"t_done":283789.90810665,"t_done_unix":1779714654.92205},{"request_id":"1270606:15:1895310:1134","session_id":"1270606","input_length":120058,"t_proxy_recv":283793.640545651,"t_decision_unix":1779714658.654487,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":119808,"affinity_cache_ratio":0.9979176731246564,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":119808,"estimated_new_tokens":250,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":119808,"new_prefill":250,"score_linear":-119808.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2301,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":118522,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-119808.0,"chosen_score_lmetric":0,"t_first_token":283793.897276335,"t_first_token_unix":1779714658.9112203,"t_done":283794.881746197,"t_done_unix":1779714659.89569},{"request_id":"1340290:29:1896119:1136","session_id":"1340290","input_length":49576,"t_proxy_recv":283796.130565554,"t_decision_unix":1779714661.144507,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":49152,"affinity_cache_ratio":0.9914474745844764,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49152,"estimated_new_tokens":424,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":49576,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":49576,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":135905,"ongoing_decode_tokens":135905,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":0,"new_prefill":49576,"score_linear":135905.0,"score_lmetric":49576},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":49576,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":0,"new_prefill":49576,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":49576,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2301,"cache_hit":49152,"new_prefill":424,"score_linear":-49152.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":49576,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49152.0,"chosen_score_lmetric":0,"t_first_token":283796.237786737,"t_first_token_unix":1779714661.2517304,"t_done":283796.485430174,"t_done_unix":1779714661.4993732},{"request_id":"1356885:13:1895311:1135","session_id":"1356885","input_length":135905,"t_proxy_recv":283793.651392985,"t_decision_unix":1779714658.6653345,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":135168,"affinity_cache_ratio":0.9945770942938081,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":135168,"estimated_new_tokens":737,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":135905,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":135905,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":135168,"new_prefill":737,"score_linear":-135168.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":6656,"new_prefill":129249,"score_linear":-6656.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":120058,"ongoing_decode_tokens":0,"pending_prefill_tokens":250,"num_requests":1,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":0,"new_prefill":135905,"score_linear":120058.0,"score_lmetric":136155},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":135905,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2301,"cache_hit":0,"new_prefill":135905,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":135905,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-135168.0,"chosen_score_lmetric":0,"t_first_token":283794.082913876,"t_first_token_unix":1779714659.0968578,"t_done":283796.518064841,"t_done_unix":1779714661.532008},{"request_id":"1340290:30:1896991:1137","session_id":"1340290","input_length":49695,"t_proxy_recv":283799.104746742,"t_decision_unix":1779714664.118688,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":49152,"affinity_cache_ratio":0.9890733474192575,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49152,"estimated_new_tokens":543,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2508,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2301,"cache_hit":49152,"new_prefill":543,"score_linear":-49152.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49152.0,"chosen_score_lmetric":0,"t_first_token":283799.213384579,"t_first_token_unix":1779714664.2273283,"t_done":283800.721864758,"t_done_unix":1779714665.735808},{"request_id":"1340290:31:1899492:1138","session_id":"1340290","input_length":49927,"t_proxy_recv":283807.488536033,"t_decision_unix":1779714672.5024774,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9947323091713902,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":263,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2508,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2302,"cache_hit":49664,"new_prefill":263,"score_linear":-49664.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":283807.579369607,"t_first_token_unix":1779714672.5933127,"t_done":283808.239738072,"t_done_unix":1779714673.2536814},{"request_id":"1270606:16:1902252:1140","session_id":"1270606","input_length":120572,"t_proxy_recv":283815.990376599,"t_decision_unix":1779714681.0043178,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":119808,"affinity_cache_ratio":0.9936635371396344,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":119808,"estimated_new_tokens":764,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":120572,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":120572,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2508,"cache_hit":0,"new_prefill":120572,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":120572,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":119808,"new_prefill":764,"score_linear":-119808.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":120572,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50078,"ongoing_decode_tokens":50078,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2302,"cache_hit":0,"new_prefill":120572,"score_linear":50078.0,"score_lmetric":120572},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":119036,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-119808.0,"chosen_score_lmetric":0,"t_first_token":283816.578049941,"t_first_token_unix":1779714681.5919933,"t_done":283817.538830594,"t_done_unix":1779714682.5527742},{"request_id":"1270606:17:1908800:1141","session_id":"1270606","input_length":121006,"t_proxy_recv":283837.586677849,"t_decision_unix":1779714702.6006193,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":120320,"affinity_cache_ratio":0.9943308596267953,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":120320,"estimated_new_tokens":686,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":121006,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":121006,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2508,"cache_hit":0,"new_prefill":121006,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":121006,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2240,"cache_hit":120320,"new_prefill":686,"score_linear":-120320.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":121006,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50078,"ongoing_decode_tokens":50078,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2302,"cache_hit":0,"new_prefill":121006,"score_linear":50078.0,"score_lmetric":121006},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":119470,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-120320.0,"chosen_score_lmetric":0,"t_first_token":283838.169179453,"t_first_token_unix":1779714703.1831229,"t_done":283838.789578857,"t_done_unix":1779714703.8035223},{"request_id":"1340290:32:1901260:1139","session_id":"1340290","input_length":50078,"t_proxy_recv":283812.904797703,"t_decision_unix":1779714677.9187388,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9917328966811774,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":414,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2508,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2302,"cache_hit":49664,"new_prefill":414,"score_linear":-49664.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":283813.026928251,"t_first_token_unix":1779714678.0408719,"t_done":283843.63944788,"t_done_unix":1779714708.6533911},{"request_id":"1270606:18:1915824:1142","session_id":"1270606","input_length":121288,"t_proxy_recv":283860.568882746,"t_decision_unix":1779714725.5828238,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":120832,"affinity_cache_ratio":0.9962403535386848,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":120832,"estimated_new_tokens":456,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2508,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":120832,"new_prefill":456,"score_linear":-120832.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2303,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":119752,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-120832.0,"chosen_score_lmetric":0,"t_first_token":283860.963465074,"t_first_token_unix":1779714725.9774084,"t_done":283861.820177379,"t_done_unix":1779714726.834121},{"request_id":"1270606:19:1920499:1143","session_id":"1270606","input_length":121560,"t_proxy_recv":283875.705993383,"t_decision_unix":1779714740.7199342,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":121344,"affinity_cache_ratio":0.9982230997038499,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":121344,"estimated_new_tokens":216,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2508,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":121344,"new_prefill":216,"score_linear":-121344.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2303,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":120024,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-121344.0,"chosen_score_lmetric":0,"t_first_token":283876.079486754,"t_first_token_unix":1779714741.093431,"t_done":283877.06242653,"t_done_unix":1779714742.0763702},{"request_id":"1270606:20:1922639:1144","session_id":"1270606","input_length":121704,"t_proxy_recv":283883.156324178,"t_decision_unix":1779714748.1702657,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":121344,"affinity_cache_ratio":0.9970420035495957,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":121344,"estimated_new_tokens":360,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2508,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":121344,"new_prefill":360,"score_linear":-121344.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2303,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":1536,"new_prefill":120168,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-121344.0,"chosen_score_lmetric":0,"t_first_token":283883.394881327,"t_first_token_unix":1779714748.4088252,"t_done":283884.71750097,"t_done_unix":1779714749.7314458},{"request_id":"1305906:10:1928316:1145","session_id":"1305906","input_length":61801,"t_proxy_recv":283901.719504884,"t_decision_unix":1779714766.733446,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9527353926311872,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":2921,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2508,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":16896,"new_prefill":44905,"score_linear":-16896.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":58880,"new_prefill":2921,"score_linear":-58880.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2303,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58880.0,"chosen_score_lmetric":0,"t_first_token":283902.774524132,"t_first_token_unix":1779714767.788468,"t_done":283903.107531462,"t_done_unix":1779714768.1214747},{"request_id":"1340290:33:1939781:1146","session_id":"1340290","input_length":53990,"t_proxy_recv":283939.462848722,"t_decision_unix":1779714804.47679,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":53760,"affinity_cache_ratio":0.9957399518429338,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":53760,"estimated_new_tokens":230,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2508,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2303,"cache_hit":53760,"new_prefill":230,"score_linear":-53760.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53760.0,"chosen_score_lmetric":0,"t_first_token":283939.559560111,"t_first_token_unix":1779714804.5735035,"t_done":283940.655525539,"t_done_unix":1779714805.6694686},{"request_id":"1340290:34:1942249:1147","session_id":"1340290","input_length":54164,"t_proxy_recv":283947.318384294,"t_decision_unix":1779714812.3323255,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":53760,"affinity_cache_ratio":0.992541171257662,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":53760,"estimated_new_tokens":404,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2508,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2303,"cache_hit":53760,"new_prefill":404,"score_linear":-53760.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53760.0,"chosen_score_lmetric":0,"t_first_token":283947.414421711,"t_first_token_unix":1779714812.428365,"t_done":283949.886551649,"t_done_unix":1779714814.9004948},{"request_id":"1356885:14:1948151:1148","session_id":"1356885","input_length":136210,"t_proxy_recv":283967.08602583,"t_decision_unix":1779714832.099967,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":135680,"affinity_cache_ratio":0.9961089494163424,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":135680,"estimated_new_tokens":530,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":136210,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":136210,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2508,"cache_hit":135680,"new_prefill":530,"score_linear":-135680.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":6656,"new_prefill":129554,"score_linear":-6656.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":136210,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":136210,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":136210,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":136210,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-135680.0,"chosen_score_lmetric":0,"t_first_token":283967.420360542,"t_first_token_unix":1779714832.434304,"t_done":283969.124387402,"t_done_unix":1779714834.1383307},{"request_id":"1340290:35:1964727:1149","session_id":"1340290","input_length":54498,"t_proxy_recv":284021.752509693,"t_decision_unix":1779714886.766451,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":54272,"affinity_cache_ratio":0.9958530588278469,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54272,"estimated_new_tokens":226,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":54272,"new_prefill":226,"score_linear":-54272.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54272.0,"chosen_score_lmetric":0,"t_first_token":284021.851230787,"t_first_token_unix":1779714886.8651743,"t_done":284025.652309028,"t_done_unix":1779714890.6662524},{"request_id":"1340290:36:1969509:1150","session_id":"1340290","input_length":54811,"t_proxy_recv":284037.554441562,"t_decision_unix":1779714902.5683827,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9995073981500063,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54784,"estimated_new_tokens":27,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2305,"cache_hit":54784,"new_prefill":27,"score_linear":-54784.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":284037.635686511,"t_first_token_unix":1779714902.64963,"t_done":284038.090354772,"t_done_unix":1779714903.1042979},{"request_id":"1340290:37:1971106:1151","session_id":"1340290","input_length":55358,"t_proxy_recv":284042.69880115,"t_decision_unix":1779714907.7127423,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9896311282922071,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54784,"estimated_new_tokens":574,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2305,"cache_hit":54784,"new_prefill":574,"score_linear":-54784.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":284043.000686668,"t_first_token_unix":1779714908.01463,"t_done":284043.459249051,"t_done_unix":1779714908.4731922},{"request_id":"1340290:38:1972289:1152","session_id":"1340290","input_length":56006,"t_proxy_recv":284046.626282068,"t_decision_unix":1779714911.6402235,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":55296,"affinity_cache_ratio":0.9873227868442667,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":55296,"estimated_new_tokens":710,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2306,"cache_hit":55296,"new_prefill":710,"score_linear":-55296.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-55296.0,"chosen_score_lmetric":0,"t_first_token":284046.933111651,"t_first_token_unix":1779714911.9470549,"t_done":284047.598882455,"t_done_unix":1779714912.6128254},{"request_id":"1340290:39:1973889:1153","session_id":"1340290","input_length":56242,"t_proxy_recv":284052.050673801,"t_decision_unix":1779714917.0646152,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":55808,"affinity_cache_ratio":0.9922833469649017,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":55808,"estimated_new_tokens":434,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2307,"cache_hit":55808,"new_prefill":434,"score_linear":-55808.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-55808.0,"chosen_score_lmetric":0,"t_first_token":284052.236500758,"t_first_token_unix":1779714917.250444,"t_done":284052.90151814,"t_done_unix":1779714917.9154613},{"request_id":"1340290:40:1975278:1154","session_id":"1340290","input_length":56416,"t_proxy_recv":284056.616051973,"t_decision_unix":1779714921.6299932,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":56320,"affinity_cache_ratio":0.998298355076574,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":56320,"estimated_new_tokens":96,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2308,"cache_hit":56320,"new_prefill":96,"score_linear":-56320.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-56320.0,"chosen_score_lmetric":0,"t_first_token":284056.73259401,"t_first_token_unix":1779714921.7465374,"t_done":284057.218884808,"t_done_unix":1779714922.232829},{"request_id":"1340290:41:1977055:1156","session_id":"1340290","input_length":57181,"t_proxy_recv":284062.550512372,"t_decision_unix":1779714927.5644536,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":56320,"affinity_cache_ratio":0.9849425508473094,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":56320,"estimated_new_tokens":861,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":57181,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":57181,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":0,"new_prefill":57181,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":67793,"ongoing_decode_tokens":0,"pending_prefill_tokens":2769,"num_requests":1,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":0,"new_prefill":57181,"score_linear":67793.0,"score_lmetric":59950},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":57181,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":57181,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2308,"cache_hit":56320,"new_prefill":861,"score_linear":-56320.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":57181,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-56320.0,"chosen_score_lmetric":0,"t_first_token":284062.965677267,"t_first_token_unix":1779714927.9796207,"t_done":284064.955720709,"t_done_unix":1779714929.9696646},{"request_id":"1277428:21:1976968:1155","session_id":"1277428","input_length":67793,"t_proxy_recv":284062.266245982,"t_decision_unix":1779714927.2801874,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9591550750077442,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":65024,"estimated_new_tokens":2769,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":67793,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":4608,"new_prefill":63185,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":0,"new_prefill":67793,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2158,"cache_hit":65024,"new_prefill":2769,"score_linear":-65024.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":67793,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":67793,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2308,"cache_hit":0,"new_prefill":67793,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":67793,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":284063.516573107,"t_first_token_unix":1779714928.5305164,"t_done":284069.415234025,"t_done_unix":1779714934.4291778},{"request_id":"1340290:42:1982198:1157","session_id":"1340290","input_length":57463,"t_proxy_recv":284079.448785262,"t_decision_unix":1779714944.462726,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":57344,"affinity_cache_ratio":0.9979291022048971,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57344,"estimated_new_tokens":119,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2164,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2310,"cache_hit":57344,"new_prefill":119,"score_linear":-57344.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57344.0,"chosen_score_lmetric":0,"t_first_token":284079.549278728,"t_first_token_unix":1779714944.5632222,"t_done":284080.043762269,"t_done_unix":1779714945.0577054},{"request_id":"1340290:43:1983901:1158","session_id":"1340290","input_length":57640,"t_proxy_recv":284084.850470782,"t_decision_unix":1779714949.8644118,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":57344,"affinity_cache_ratio":0.9948646773074254,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57344,"estimated_new_tokens":296,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2164,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2310,"cache_hit":57344,"new_prefill":296,"score_linear":-57344.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57344.0,"chosen_score_lmetric":0,"t_first_token":284085.00718086,"t_first_token_unix":1779714950.0211246,"t_done":284085.523589446,"t_done_unix":1779714950.5375347},{"request_id":"1340290:44:1986230:1159","session_id":"1340290","input_length":57809,"t_proxy_recv":284092.932285033,"t_decision_unix":1779714957.9462264,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":57344,"affinity_cache_ratio":0.9919562697849815,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57344,"estimated_new_tokens":465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2164,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2310,"cache_hit":57344,"new_prefill":465,"score_linear":-57344.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57344.0,"chosen_score_lmetric":0,"t_first_token":284093.091666584,"t_first_token_unix":1779714958.1056101,"t_done":284093.808104667,"t_done_unix":1779714958.8220482},{"request_id":"1277428:21:1989132:1160","session_id":"1277428","input_length":69987,"t_proxy_recv":284102.351277342,"t_decision_unix":1779714967.3652189,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":68096,"affinity_cache_ratio":0.9729806964150485,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":68096,"estimated_new_tokens":1891,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":69987,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":4608,"new_prefill":65379,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":0,"new_prefill":69987,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2164,"cache_hit":68096,"new_prefill":1891,"score_linear":-68096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":69987,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":69987,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2311,"cache_hit":0,"new_prefill":69987,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":69987,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-68096.0,"chosen_score_lmetric":0,"t_first_token":284103.117596662,"t_first_token_unix":1779714968.1315398,"t_done":284103.464343711,"t_done_unix":1779714968.4782867},{"request_id":"1277428:22:1990436:1161","session_id":"1277428","input_length":70055,"t_proxy_recv":284106.892976014,"t_decision_unix":1779714971.906917,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":69632,"affinity_cache_ratio":0.993961887088716,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":69632,"estimated_new_tokens":423,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":70055,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":4608,"new_prefill":65447,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":0,"new_prefill":70055,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":69632,"new_prefill":423,"score_linear":-69632.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":70055,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":70055,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2311,"cache_hit":0,"new_prefill":70055,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":70055,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-69632.0,"chosen_score_lmetric":0,"t_first_token":284107.012038546,"t_first_token_unix":1779714972.0259821,"t_done":284107.422024992,"t_done_unix":1779714972.4359682},{"request_id":"1340290:45:1991694:1162","session_id":"1340290","input_length":58053,"t_proxy_recv":284110.905810438,"t_decision_unix":1779714975.9197512,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":57856,"affinity_cache_ratio":0.9966065491878111,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57856,"estimated_new_tokens":197,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2311,"cache_hit":57856,"new_prefill":197,"score_linear":-57856.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57856.0,"chosen_score_lmetric":0,"t_first_token":284111.098102362,"t_first_token_unix":1779714976.1120458,"t_done":284111.35171274,"t_done_unix":1779714976.365656},{"request_id":"1340290:46:1992980:1163","session_id":"1340290","input_length":58122,"t_proxy_recv":284115.130487647,"t_decision_unix":1779714980.1444285,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":57856,"affinity_cache_ratio":0.9954234197033826,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57856,"estimated_new_tokens":266,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2311,"cache_hit":57856,"new_prefill":266,"score_linear":-57856.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57856.0,"chosen_score_lmetric":0,"t_first_token":284115.231842237,"t_first_token_unix":1779714980.2457862,"t_done":284115.650232407,"t_done_unix":1779714980.6641755},{"request_id":"1340290:47:1995358:1164","session_id":"1340290","input_length":58307,"t_proxy_recv":284122.989372855,"t_decision_unix":1779714988.0033143,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":57856,"affinity_cache_ratio":0.9922650796645343,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57856,"estimated_new_tokens":451,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2311,"cache_hit":57856,"new_prefill":451,"score_linear":-57856.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57856.0,"chosen_score_lmetric":0,"t_first_token":284123.129759974,"t_first_token_unix":1779714988.1437035,"t_done":284123.383287392,"t_done_unix":1779714988.3972306},{"request_id":"1340290:48:1996945:1165","session_id":"1340290","input_length":58339,"t_proxy_recv":284128.293989626,"t_decision_unix":1779714993.3079307,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":57856,"affinity_cache_ratio":0.9917208042647285,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57856,"estimated_new_tokens":483,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2311,"cache_hit":57856,"new_prefill":483,"score_linear":-57856.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57856.0,"chosen_score_lmetric":0,"t_first_token":284128.373680036,"t_first_token_unix":1779714993.3876233,"t_done":284130.404661241,"t_done_unix":1779714995.4186046},{"request_id":"1356885:15:1998292:1166","session_id":"1356885","input_length":136653,"t_proxy_recv":284132.769193895,"t_decision_unix":1779714997.783135,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":136192,"affinity_cache_ratio":0.9966264919174844,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":136192,"estimated_new_tokens":461,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":136653,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":136653,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":136192,"new_prefill":461,"score_linear":-136192.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":6656,"new_prefill":129997,"score_linear":-6656.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":136653,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":136653,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2312,"cache_hit":0,"new_prefill":136653,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":136653,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-136192.0,"chosen_score_lmetric":0,"t_first_token":284133.43190311,"t_first_token_unix":1779714998.4458466,"t_done":284135.108382773,"t_done_unix":1779715000.122326},{"request_id":"1340290:48:1999591:1167","session_id":"1340290","input_length":58622,"t_proxy_recv":284137.313081473,"t_decision_unix":1779715002.3270228,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":58368,"affinity_cache_ratio":0.9956671556753437,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58368,"estimated_new_tokens":254,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2312,"cache_hit":58368,"new_prefill":254,"score_linear":-58368.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58368.0,"chosen_score_lmetric":0,"t_first_token":284137.416433775,"t_first_token_unix":1779715002.4303772,"t_done":284138.014723656,"t_done_unix":1779715003.0286667},{"request_id":"1340290:48:2001313:1168","session_id":"1340290","input_length":58808,"t_proxy_recv":284142.779538398,"t_decision_unix":1779715007.7934797,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":58368,"affinity_cache_ratio":0.9925180247585362,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58368,"estimated_new_tokens":440,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2312,"cache_hit":58368,"new_prefill":440,"score_linear":-58368.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58368.0,"chosen_score_lmetric":0,"t_first_token":284142.952034064,"t_first_token_unix":1779715007.9659784,"t_done":284143.458319921,"t_done_unix":1779715008.472263},{"request_id":"1340290:48:2003195:1169","session_id":"1340290","input_length":58977,"t_proxy_recv":284149.104687207,"t_decision_unix":1779715014.118628,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":58368,"affinity_cache_ratio":0.989673940688743,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58368,"estimated_new_tokens":609,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2312,"cache_hit":58368,"new_prefill":609,"score_linear":-58368.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58368.0,"chosen_score_lmetric":0,"t_first_token":284149.275838722,"t_first_token_unix":1779715014.2897823,"t_done":284150.155088562,"t_done_unix":1779715015.1690328},{"request_id":"1340290:48:2005449:1170","session_id":"1340290","input_length":59220,"t_proxy_recv":284156.493365998,"t_decision_unix":1779715021.507307,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":58880,"affinity_cache_ratio":0.994258696386356,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":340,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2313,"cache_hit":58880,"new_prefill":340,"score_linear":-58880.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58880.0,"chosen_score_lmetric":0,"t_first_token":284156.649075295,"t_first_token_unix":1779715021.6630197,"t_done":284157.484403454,"t_done_unix":1779715022.4983466},{"request_id":"1340290:48:2007287:1171","session_id":"1340290","input_length":59487,"t_proxy_recv":284162.618935697,"t_decision_unix":1779715027.6328769,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9897960899019954,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":607,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2313,"cache_hit":58880,"new_prefill":607,"score_linear":-58880.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58880.0,"chosen_score_lmetric":0,"t_first_token":284162.819474093,"t_first_token_unix":1779715027.8334181,"t_done":284163.203480318,"t_done_unix":1779715028.2174258},{"request_id":"1340290:48:2008380:1173","session_id":"1340290","input_length":60935,"t_proxy_recv":284166.863600306,"t_decision_unix":1779715031.8775413,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":59392,"affinity_cache_ratio":0.9746779355050463,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":59392,"estimated_new_tokens":1543,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66506,"ongoing_decode_tokens":0,"pending_prefill_tokens":5066,"num_requests":1,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":60935,"score_linear":66506.0,"score_lmetric":66001},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":59392,"new_prefill":1543,"score_linear":-59392.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-59392.0,"chosen_score_lmetric":0,"t_first_token":284167.554239704,"t_first_token_unix":1779715032.5681837,"t_done":284168.236538624,"t_done_unix":1779715033.2504823},{"request_id":"1305906:11:2008110:1172","session_id":"1305906","input_length":66506,"t_proxy_recv":284165.717061607,"t_decision_unix":1779715030.7310026,"policy":"unified","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":61440,"affinity_cache_ratio":0.9238264216762397,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":61440,"estimated_new_tokens":5066,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":16896,"new_prefill":49610,"score_linear":-16896.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":61440,"new_prefill":5066,"score_linear":-61440.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-61440.0,"chosen_score_lmetric":0,"t_first_token":284167.705177283,"t_first_token_unix":1779715032.7191207,"t_done":284170.057163654,"t_done_unix":1779715035.0711079},{"request_id":"1340290:48:2010180:1174","session_id":"1340290","input_length":61080,"t_proxy_recv":284172.580488563,"t_decision_unix":1779715037.5944293,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":60928,"affinity_cache_ratio":0.9975114603798297,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":60928,"estimated_new_tokens":152,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":60928,"new_prefill":152,"score_linear":-60928.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-60928.0,"chosen_score_lmetric":0,"t_first_token":284172.73304502,"t_first_token_unix":1779715037.7469893,"t_done":284173.751552735,"t_done_unix":1779715038.7654967},{"request_id":"1340290:48:2012871:1175","session_id":"1340290","input_length":61365,"t_proxy_recv":284181.257612046,"t_decision_unix":1779715046.271544,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":60928,"affinity_cache_ratio":0.9928786767701459,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":60928,"estimated_new_tokens":437,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":60928,"new_prefill":437,"score_linear":-60928.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-60928.0,"chosen_score_lmetric":0,"t_first_token":284181.461547305,"t_first_token_unix":1779715046.475491,"t_done":284181.985054432,"t_done_unix":1779715046.9989982},{"request_id":"1340290:48:2014153:1176","session_id":"1340290","input_length":61518,"t_proxy_recv":284185.656591045,"t_decision_unix":1779715050.670532,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":60928,"affinity_cache_ratio":0.9904093110959394,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":60928,"estimated_new_tokens":590,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":60928,"new_prefill":590,"score_linear":-60928.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-60928.0,"chosen_score_lmetric":0,"t_first_token":284185.786509738,"t_first_token_unix":1779715050.8004537,"t_done":284186.286489868,"t_done_unix":1779715051.300433},{"request_id":"1340290:48:2015982:1177","session_id":"1340290","input_length":69113,"t_proxy_recv":284191.756623385,"t_decision_unix":1779715056.7705638,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":61440,"affinity_cache_ratio":0.8889789185826111,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":61440,"estimated_new_tokens":7673,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2318,"cache_hit":61440,"new_prefill":7673,"score_linear":-61440.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-61440.0,"chosen_score_lmetric":0,"t_first_token":284194.999271003,"t_first_token_unix":1779715060.0132153,"t_done":284197.450561469,"t_done_unix":1779715062.4645054},{"request_id":"1340290:48:2018973:1178","session_id":"1340290","input_length":69426,"t_proxy_recv":284201.540558413,"t_decision_unix":1779715066.5544994,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":69120,"affinity_cache_ratio":0.9955924293492352,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":69120,"estimated_new_tokens":306,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2333,"cache_hit":69120,"new_prefill":306,"score_linear":-69120.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-69120.0,"chosen_score_lmetric":0,"t_first_token":284201.660515274,"t_first_token_unix":1779715066.6744592,"t_done":284202.251829266,"t_done_unix":1779715067.2657733},{"request_id":"1340290:48:2020779:1179","session_id":"1340290","input_length":69601,"t_proxy_recv":284207.595680243,"t_decision_unix":1779715072.609621,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":69120,"affinity_cache_ratio":0.9930891797531645,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":69120,"estimated_new_tokens":481,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2333,"cache_hit":69120,"new_prefill":481,"score_linear":-69120.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-69120.0,"chosen_score_lmetric":0,"t_first_token":284207.780400051,"t_first_token_unix":1779715072.794344,"t_done":284208.353427855,"t_done_unix":1779715073.3673713},{"request_id":"1340290:48:2023753:1180","session_id":"1340290","input_length":69951,"t_proxy_recv":284217.212722628,"t_decision_unix":1779715082.2266638,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":69632,"affinity_cache_ratio":0.9954396649082929,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":69632,"estimated_new_tokens":319,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2334,"cache_hit":69632,"new_prefill":319,"score_linear":-69632.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-69632.0,"chosen_score_lmetric":0,"t_first_token":284217.445994901,"t_first_token_unix":1779715082.4599385,"t_done":284218.340267943,"t_done_unix":1779715083.3542118},{"request_id":"1340290:48:2026751:1181","session_id":"1340290","input_length":70342,"t_proxy_recv":284227.493191695,"t_decision_unix":1779715092.5071325,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":69632,"affinity_cache_ratio":0.989906457024253,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":69632,"estimated_new_tokens":710,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2334,"cache_hit":69632,"new_prefill":710,"score_linear":-69632.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-69632.0,"chosen_score_lmetric":0,"t_first_token":284227.729355408,"t_first_token_unix":1779715092.7432992,"t_done":284228.257575435,"t_done_unix":1779715093.271519},{"request_id":"1340290:48:2028652:1182","session_id":"1340290","input_length":70503,"t_proxy_recv":284233.744340659,"t_decision_unix":1779715098.7582812,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":70144,"affinity_cache_ratio":0.9949080180985206,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":70144,"estimated_new_tokens":359,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2335,"cache_hit":70144,"new_prefill":359,"score_linear":-70144.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-70144.0,"chosen_score_lmetric":0,"t_first_token":284233.92859811,"t_first_token_unix":1779715098.9425418,"t_done":284234.832047734,"t_done_unix":1779715099.8464153},{"request_id":"1340290:48:2033065:1183","session_id":"1340290","input_length":70851,"t_proxy_recv":284248.469264129,"t_decision_unix":1779715113.483205,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":70144,"affinity_cache_ratio":0.9900213123315126,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":70144,"estimated_new_tokens":707,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2335,"cache_hit":70144,"new_prefill":707,"score_linear":-70144.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-70144.0,"chosen_score_lmetric":0,"t_first_token":284248.714121678,"t_first_token_unix":1779715113.7280655,"t_done":284249.356837159,"t_done_unix":1779715114.3707807},{"request_id":"1340290:48:2034908:1184","session_id":"1340290","input_length":71067,"t_proxy_recv":284254.696244728,"t_decision_unix":1779715119.7101858,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":70656,"affinity_cache_ratio":0.9942167250622652,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":70656,"estimated_new_tokens":411,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2336,"cache_hit":70656,"new_prefill":411,"score_linear":-70656.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-70656.0,"chosen_score_lmetric":0,"t_first_token":284254.864586793,"t_first_token_unix":1779715119.8785307,"t_done":284255.381350621,"t_done_unix":1779715120.395294},{"request_id":"1258908:2:2035438:1185","session_id":"1258908","input_length":7844,"t_proxy_recv":284256.50151539,"t_decision_unix":1779715121.5154562,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":7680,"affinity_cache_ratio":0.9790922998470168,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":164,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":7680,"new_prefill":164,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2336,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":284256.529781488,"t_first_token_unix":1779715121.5437253,"t_done":284257.56710638,"t_done_unix":1779715122.5810502},{"request_id":"1258908:3:1388683:699","session_id":"1258908","input_length":9120,"t_proxy_recv":284257.570586684,"t_decision_unix":1779715122.5845284,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":7680,"affinity_cache_ratio":0.8421052631578947,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":1440,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2167,"cache_hit":7680,"new_prefill":1440,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2336,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":284257.713982533,"t_first_token_unix":1779715122.7279263,"t_done":284258.399538248,"t_done_unix":1779715123.413482},{"request_id":"1340290:48:2036994:1186","session_id":"1340290","input_length":71240,"t_proxy_recv":284261.63829131,"t_decision_unix":1779715126.6522322,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":70656,"affinity_cache_ratio":0.9918023582257159,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":70656,"estimated_new_tokens":584,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2170,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2336,"cache_hit":70656,"new_prefill":584,"score_linear":-70656.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-70656.0,"chosen_score_lmetric":0,"t_first_token":284261.828195226,"t_first_token_unix":1779715126.8421388,"t_done":284262.551102887,"t_done_unix":1779715127.5650463},{"request_id":"1340290:48:2040473:1187","session_id":"1340290","input_length":71545,"t_proxy_recv":284273.155780165,"t_decision_unix":1779715138.1697214,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9947305891397022,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71168,"estimated_new_tokens":377,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2170,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2337,"cache_hit":71168,"new_prefill":377,"score_linear":-71168.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-71168.0,"chosen_score_lmetric":0,"t_first_token":284273.39326864,"t_first_token_unix":1779715138.4072123,"t_done":284274.543402741,"t_done_unix":1779715139.557346},{"request_id":"1340290:48:2043771:1188","session_id":"1340290","input_length":71867,"t_proxy_recv":284284.173062125,"t_decision_unix":1779715149.1870034,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9902737000292207,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71168,"estimated_new_tokens":699,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2170,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2337,"cache_hit":71168,"new_prefill":699,"score_linear":-71168.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-71168.0,"chosen_score_lmetric":0,"t_first_token":284284.40659892,"t_first_token_unix":1779715149.4205425,"t_done":284284.986901733,"t_done_unix":1779715150.000845},{"request_id":"1340290:48:2045403:1189","session_id":"1340290","input_length":72025,"t_proxy_recv":284289.692579978,"t_decision_unix":1779715154.706521,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":71680,"affinity_cache_ratio":0.995209996528983,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71680,"estimated_new_tokens":345,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2170,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":71680,"new_prefill":345,"score_linear":-71680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-71680.0,"chosen_score_lmetric":0,"t_first_token":284289.835994459,"t_first_token_unix":1779715154.849938,"t_done":284290.52545353,"t_done_unix":1779715155.5393965},{"request_id":"1340290:48:2047167:1190","session_id":"1340290","input_length":72211,"t_proxy_recv":284295.456511635,"t_decision_unix":1779715160.4704525,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":71680,"affinity_cache_ratio":0.9926465496946448,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71680,"estimated_new_tokens":531,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2170,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2338,"cache_hit":71680,"new_prefill":531,"score_linear":-71680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-71680.0,"chosen_score_lmetric":0,"t_first_token":284295.644957538,"t_first_token_unix":1779715160.6589012,"t_done":284296.198417138,"t_done_unix":1779715161.2123606},{"request_id":"1340290:48:2048793:1191","session_id":"1340290","input_length":72777,"t_proxy_recv":284300.8907739,"t_decision_unix":1779715165.9047148,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":72192,"affinity_cache_ratio":0.9919617461560658,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":72192,"estimated_new_tokens":585,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2170,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2339,"cache_hit":72192,"new_prefill":585,"score_linear":-72192.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-72192.0,"chosen_score_lmetric":0,"t_first_token":284301.270048717,"t_first_token_unix":1779715166.2839928,"t_done":284301.795555084,"t_done_unix":1779715166.809499},{"request_id":"1340290:48:2050838:1192","session_id":"1340290","input_length":72952,"t_proxy_recv":284308.111741791,"t_decision_unix":1779715173.1256826,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":72704,"affinity_cache_ratio":0.9966005044412765,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":72704,"estimated_new_tokens":248,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2170,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2340,"cache_hit":72704,"new_prefill":248,"score_linear":-72704.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-72704.0,"chosen_score_lmetric":0,"t_first_token":284308.300379872,"t_first_token_unix":1779715173.3143234,"t_done":284308.664339793,"t_done_unix":1779715173.678283},{"request_id":"1340290:48:2052523:1193","session_id":"1340290","input_length":73208,"t_proxy_recv":284313.678970979,"t_decision_unix":1779715178.692912,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":72704,"affinity_cache_ratio":0.9931155065020216,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":72704,"estimated_new_tokens":504,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2170,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2340,"cache_hit":72704,"new_prefill":504,"score_linear":-72704.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-72704.0,"chosen_score_lmetric":0,"t_first_token":284313.917793802,"t_first_token_unix":1779715178.9317381,"t_done":284314.704270064,"t_done_unix":1779715179.7182136},{"request_id":"1340290:48:2055365:1194","session_id":"1340290","input_length":73454,"t_proxy_recv":284323.391675049,"t_decision_unix":1779715188.405616,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":73216,"affinity_cache_ratio":0.9967598769297792,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73216,"estimated_new_tokens":238,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2170,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2341,"cache_hit":73216,"new_prefill":238,"score_linear":-73216.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73216.0,"chosen_score_lmetric":0,"t_first_token":284323.626568174,"t_first_token_unix":1779715188.6405118,"t_done":284324.05322884,"t_done_unix":1779715189.0671725},{"request_id":"1340290:48:2057042:1195","session_id":"1340290","input_length":73636,"t_proxy_recv":284328.765913816,"t_decision_unix":1779715193.7798545,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":73216,"affinity_cache_ratio":0.9942962681297192,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73216,"estimated_new_tokens":420,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2170,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2341,"cache_hit":73216,"new_prefill":420,"score_linear":-73216.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73216.0,"chosen_score_lmetric":0,"t_first_token":284328.93689493,"t_first_token_unix":1779715193.9508388,"t_done":284329.209417806,"t_done_unix":1779715194.2233615},{"request_id":"1340290:48:2058614:1196","session_id":"1340290","input_length":73653,"t_proxy_recv":284333.906155292,"t_decision_unix":1779715198.9200964,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":73216,"affinity_cache_ratio":0.9940667725686666,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73216,"estimated_new_tokens":437,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2170,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2341,"cache_hit":73216,"new_prefill":437,"score_linear":-73216.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73216.0,"chosen_score_lmetric":0,"t_first_token":284334.029308223,"t_first_token_unix":1779715199.0432518,"t_done":284335.766765749,"t_done_unix":1779715200.780709},{"request_id":"1340290:48:2061834:1197","session_id":"1340290","input_length":73858,"t_proxy_recv":284344.057388535,"t_decision_unix":1779715209.0713294,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":73728,"affinity_cache_ratio":0.9982398656882125,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73728,"estimated_new_tokens":130,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2170,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2342,"cache_hit":73728,"new_prefill":130,"score_linear":-73728.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73728.0,"chosen_score_lmetric":0,"t_first_token":284344.179898669,"t_first_token_unix":1779715209.1938426,"t_done":284344.732721231,"t_done_unix":1779715209.746665},{"request_id":"1258908:3:2063441:1198","session_id":"1258908","input_length":9359,"t_proxy_recv":284349.096032585,"t_decision_unix":1779715214.1099737,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":9216,"affinity_cache_ratio":0.9847205898066033,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9216,"estimated_new_tokens":143,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2170,"cache_hit":9216,"new_prefill":143,"score_linear":-9216.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2342,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9216.0,"chosen_score_lmetric":0,"t_first_token":284349.145799481,"t_first_token_unix":1779715214.1597433,"t_done":284350.232041805,"t_done_unix":1779715215.2459848},{"request_id":"1340290:48:2063837:1199","session_id":"1340290","input_length":74011,"t_proxy_recv":284350.449482632,"t_decision_unix":1779715215.4634237,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":73728,"affinity_cache_ratio":0.9961762440718271,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73728,"estimated_new_tokens":283,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":74011,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":74011,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":74011,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":9740,"ongoing_decode_tokens":9740,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2170,"cache_hit":0,"new_prefill":74011,"score_linear":9740.0,"score_lmetric":74011},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":74011,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":74011,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2342,"cache_hit":73728,"new_prefill":283,"score_linear":-73728.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":74011,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73728.0,"chosen_score_lmetric":0,"t_first_token":284350.596069494,"t_first_token_unix":1779715215.6100135,"t_done":284351.177274432,"t_done_unix":1779715216.1912186},{"request_id":"1258908:4:1553508:958","session_id":"1258908","input_length":9740,"t_proxy_recv":284350.2358405,"t_decision_unix":1779715215.2497818,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":9216,"affinity_cache_ratio":0.9462012320328542,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9216,"estimated_new_tokens":524,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2170,"cache_hit":9216,"new_prefill":524,"score_linear":-9216.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2342,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9216.0,"chosen_score_lmetric":0,"t_first_token":284350.29434835,"t_first_token_unix":1779715215.3082922,"t_done":284352.224578371,"t_done_unix":1779715217.2385223},{"request_id":"1258908:5:1617181:988","session_id":"1258908","input_length":10201,"t_proxy_recv":284352.228057727,"t_decision_unix":1779715217.2419991,"policy":"unified","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":9728,"affinity_cache_ratio":0.9536319968630527,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9728,"estimated_new_tokens":473,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2171,"cache_hit":9728,"new_prefill":473,"score_linear":-9728.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2342,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9728.0,"chosen_score_lmetric":0,"t_first_token":284352.278901963,"t_first_token_unix":1779715217.292847,"t_done":284354.363660574,"t_done_unix":1779715219.3776047},{"request_id":"1340290:48:2066647:1200","session_id":"1340290","input_length":74557,"t_proxy_recv":284359.890170264,"t_decision_unix":1779715224.9041107,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":73728,"affinity_cache_ratio":0.9888809903831968,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73728,"estimated_new_tokens":829,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2172,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2342,"cache_hit":73728,"new_prefill":829,"score_linear":-73728.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73728.0,"chosen_score_lmetric":0,"t_first_token":284360.276377716,"t_first_token_unix":1779715225.2903214,"t_done":284361.524242195,"t_done_unix":1779715226.538186},{"request_id":"1340290:48:2071598:1201","session_id":"1340290","input_length":74863,"t_proxy_recv":284376.541125978,"t_decision_unix":1779715241.5550668,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":74240,"affinity_cache_ratio":0.991678132054553,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":74240,"estimated_new_tokens":623,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2172,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2343,"cache_hit":74240,"new_prefill":623,"score_linear":-74240.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-74240.0,"chosen_score_lmetric":0,"t_first_token":284376.781969554,"t_first_token_unix":1779715241.7959135,"t_done":284377.597681049,"t_done_unix":1779715242.6116245},{"request_id":"1340290:48:2076822:1202","session_id":"1340290","input_length":75113,"t_proxy_recv":284394.061530473,"t_decision_unix":1779715259.0754714,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":74752,"affinity_cache_ratio":0.9951939078455128,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":74752,"estimated_new_tokens":361,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2172,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2344,"cache_hit":74752,"new_prefill":361,"score_linear":-74752.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-74752.0,"chosen_score_lmetric":0,"t_first_token":284394.303039012,"t_first_token_unix":1779715259.3169825,"t_done":284394.744688404,"t_done_unix":1779715259.758632},{"request_id":"1340290:48:2078461:1203","session_id":"1340290","input_length":75299,"t_proxy_recv":284399.347393951,"t_decision_unix":1779715264.3613348,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":74752,"affinity_cache_ratio":0.9927356272991673,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":74752,"estimated_new_tokens":547,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2172,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2344,"cache_hit":74752,"new_prefill":547,"score_linear":-74752.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-74752.0,"chosen_score_lmetric":0,"t_first_token":284399.598073651,"t_first_token_unix":1779715264.6120174,"t_done":284399.88068581,"t_done_unix":1779715264.8946295},{"request_id":"1340290:48:2080016:1204","session_id":"1340290","input_length":75338,"t_proxy_recv":284404.640348328,"t_decision_unix":1779715269.6542895,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":75264,"affinity_cache_ratio":0.9990177599617723,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75264,"estimated_new_tokens":74,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2172,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2345,"cache_hit":75264,"new_prefill":74,"score_linear":-75264.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75264.0,"chosen_score_lmetric":0,"t_first_token":284404.745877369,"t_first_token_unix":1779715269.7598214,"t_done":284405.123446237,"t_done_unix":1779715270.1373897},{"request_id":"1340290:48:2082644:1206","session_id":"1340290","input_length":75439,"t_proxy_recv":284413.315245141,"t_decision_unix":1779715278.329186,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":75264,"affinity_cache_ratio":0.9976802449661316,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75264,"estimated_new_tokens":175,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":75439,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":75439,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":75439,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2172,"cache_hit":0,"new_prefill":75439,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":114971,"ongoing_decode_tokens":0,"pending_prefill_tokens":61211,"num_requests":1,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":75439,"score_linear":114971.0,"score_lmetric":136650},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":75439,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2345,"cache_hit":75264,"new_prefill":175,"score_linear":-75264.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":75439,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75264.0,"chosen_score_lmetric":0,"t_first_token":284413.473582974,"t_first_token_unix":1779715278.4875267,"t_done":284414.465079984,"t_done_unix":1779715279.4790235},{"request_id":"1356885:16:2084239:1207","session_id":"1356885","input_length":136991,"t_proxy_recv":284418.412685889,"t_decision_unix":1779715283.4266264,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":136704,"affinity_cache_ratio":0.9979049718594653,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":136704,"estimated_new_tokens":287,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":136991,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":136991,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":136704,"new_prefill":287,"score_linear":-136704.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2172,"cache_hit":6656,"new_prefill":130335,"score_linear":-6656.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":114971,"ongoing_decode_tokens":0,"pending_prefill_tokens":61211,"num_requests":1,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":136991,"score_linear":114971.0,"score_lmetric":198202},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":136991,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2345,"cache_hit":0,"new_prefill":136991,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":136991,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-136704.0,"chosen_score_lmetric":0,"t_first_token":284418.851992555,"t_first_token_unix":1779715283.8659368,"t_done":284421.6952965,"t_done_unix":1779715286.7092404},{"request_id":"1340290:48:2085626:1208","session_id":"1340290","input_length":77337,"t_proxy_recv":284423.392039551,"t_decision_unix":1779715288.4059806,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":75264,"affinity_cache_ratio":0.9731952364327553,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75264,"estimated_new_tokens":2073,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":77337,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":77337,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2511,"cache_hit":0,"new_prefill":77337,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2172,"cache_hit":0,"new_prefill":77337,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":114971,"ongoing_decode_tokens":0,"pending_prefill_tokens":61211,"num_requests":1,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":77337,"score_linear":114971.0,"score_lmetric":138548},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":77337,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2345,"cache_hit":75264,"new_prefill":2073,"score_linear":-75264.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":77337,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75264.0,"chosen_score_lmetric":0,"t_first_token":284424.37271358,"t_first_token_unix":1779715289.3866572,"t_done":284424.882728594,"t_done_unix":1779715289.8966722},{"request_id":"1340290:48:2087584:1209","session_id":"1340290","input_length":78683,"t_proxy_recv":284429.985464925,"t_decision_unix":1779715294.9994001,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":77312,"affinity_cache_ratio":0.9825756516655441,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":77312,"estimated_new_tokens":1371,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":78683,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":78683,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2511,"cache_hit":0,"new_prefill":78683,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2172,"cache_hit":0,"new_prefill":78683,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":114971,"ongoing_decode_tokens":0,"pending_prefill_tokens":61211,"num_requests":1,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":78683,"score_linear":114971.0,"score_lmetric":139894},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":78683,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":77312,"new_prefill":1371,"score_linear":-77312.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":78683,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-77312.0,"chosen_score_lmetric":0,"t_first_token":284430.832917065,"t_first_token_unix":1779715295.8468611,"t_done":284432.305702203,"t_done_unix":1779715297.3196461},{"request_id":"1340290:48:2092759:1211","session_id":"1340290","input_length":79023,"t_proxy_recv":284447.252359103,"t_decision_unix":1779715312.2663002,"policy":"unified","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":78336,"affinity_cache_ratio":0.9913063285372613,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":78336,"estimated_new_tokens":687,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":79023,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":79023,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2511,"cache_hit":0,"new_prefill":79023,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2172,"cache_hit":0,"new_prefill":79023,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":114971,"ongoing_decode_tokens":0,"pending_prefill_tokens":61211,"num_requests":1,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":79023,"score_linear":114971.0,"score_lmetric":140234},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":79023,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2351,"cache_hit":78336,"new_prefill":687,"score_linear":-78336.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":79023,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-78336.0,"chosen_score_lmetric":0,"t_first_token":284447.505786647,"t_first_token_unix":1779715312.5197299,"t_done":284448.797364692,"t_done_unix":1779715313.8113084},{"request_id":"1382086:2:2082052:1205","session_id":"1382086","input_length":114971,"t_proxy_recv":284411.301675465,"t_decision_unix":1779715276.3156166,"policy":"unified","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":53760,"affinity_cache_ratio":0.467596176427099,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":53760,"estimated_new_tokens":61211,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2172,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":53760,"new_prefill":61211,"score_linear":-53760.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2345,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53760.0,"chosen_score_lmetric":0,"t_first_token":284453.933979099,"t_first_token_unix":1779715318.947923,"t_done":284455.122721864,"t_done_unix":1779715320.1366656},{"request_id":"1382086:3:2089879:1210","session_id":"1382086","input_length":115184,"t_proxy_recv":284455.134185327,"t_decision_unix":1779715320.1481264,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":114688,"affinity_cache_ratio":0.9956938463675511,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":114688,"estimated_new_tokens":496,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2511,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2172,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2361,"cache_hit":114688,"new_prefill":496,"score_linear":-114688.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2352,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-114688.0,"chosen_score_lmetric":0,"t_first_token":284455.419558659,"t_first_token_unix":1779715320.4335022,"t_done":284455.934799583,"t_done_unix":1779715320.9487429},{"request_id":"1382086:4:2092953:1212","session_id":"1382086","input_length":115246,"t_proxy_recv":284455.953672144,"t_decision_unix":1779715320.9676135,"policy":"unified","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":115200,"affinity_cache_ratio":0.9996008538257293,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":115200,"estimated_new_tokens":46,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2511,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2172,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2362,"cache_hit":115200,"new_prefill":46,"score_linear":-115200.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2352,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-115200.0,"chosen_score_lmetric":0,"t_first_token":284456.127072707,"t_first_token_unix":1779715321.141016,"t_done":284463.238389872,"t_done_unix":1779715328.2523336},{"request_id":"1356885:17:2097011:1213","session_id":"1356885","input_length":137546,"t_proxy_recv":284461.126955431,"t_decision_unix":1779715326.1408968,"policy":"unified","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":137216,"affinity_cache_ratio":0.9976008026405712,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":137216,"estimated_new_tokens":330,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1658,"cache_hit":0,"new_prefill":137546,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":137546,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2511,"cache_hit":137216,"new_prefill":330,"score_linear":-137216.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2172,"cache_hit":6656,"new_prefill":130890,"score_linear":-6656.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":115246,"ongoing_decode_tokens":115246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2362,"cache_hit":0,"new_prefill":137546,"score_linear":115246.0,"score_lmetric":137546},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2403,"cache_hit":0,"new_prefill":137546,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2352,"cache_hit":0,"new_prefill":137546,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":137546,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-137216.0,"chosen_score_lmetric":0,"t_first_token":284461.788064248,"t_first_token_unix":1779715326.8020074,"t_done":284464.48498195,"t_done_unix":1779715329.4989252}]
Reference in New Issue View Git Blame Copy Permalink