AI RESEARCH
A Policy-Driven Runtime Layer for Agentic LLM Serving
arXiv CS.AI
•
ArXi:2605.27744v1 Announce Type: new Multi-agent LLM systems have become the dominant production workload, but the serving stack was not built for them. The agent framework above knows agent identities, role, schemas, and dispatch structure but never sees an engine-level event; the serving engine below sees every event but knows nothing about agents. A surprising number of cross-cutting policies depend on both: prefix caching, batch shaping, speculative execution, fairness, tool-result memoization, safety enforcement, and more.