Cost-Aware Query Routing in RAG: Empirical Analysis of Retrieval Depth Tradeoffs

ArXi:2606.02581v1 Announce Type: cross Retrieval-augmented generation (RAG) faces a fundamental three-way tension: deeper retrieval improves factual grounding but inflates token costs and end-to-end latency. Static retrieval configurations cannot resolve this tension across heterogeneous query workloads -- simple definitional queries waste budget on unnecessary context, while complex analytical prompts are underserved by shallow retrieval. This paper