AI RESEARCH
NetKV: Network-Aware Decode Instance Selection for Disaggregated LLM Inference
arXiv CS.AI
•
ArXi:2606.03910v1 Announce Type: cross Disaggregated LLM inference forces the KV cache to traverse the datacenter network before decoding begins, so transfer time enters directly into the Time to First Token (TTFT) budget. Current schedulers route on compute load and prefix-cache locality alone, ignoring the topological distance and dynamic congestion between prefill and decode instances.