Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning

ArXi:2605.20730v1 Announce Type: new In-context learning (ICL) allows large language models (LLMs) to adapt to new tasks through nstrations, yet it suffers from escalating inference costs as context length increases. While task vectors offer a promising alternative by compressing nstrations into compact hidden-state representations, their quality has been evaluated only through downstream task accuracy. This indirect criterion provides limited insight into how to design effective task vector extraction methods.