Don't be so Stief! Learning KV Cache low-rank approximation over the Stiefel manifold

ArXi:2601.21686v2 Announce Type: replace Key-value (KV) caching enables fast autoregressive decoding but at long contexts becomes a dominant bottleneck in High Bandwidth Memory (HBM) capacity and bandwidth. A common mitigation is to compress cached keys and values by projecting per-head matrices to a lower rank, storing only the projections in the HBM. However, existing post-