AI RESEARCH
Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behavior
arXiv CS.LG
•
ArXi:2605.26797v1 Announce Type: new We study Latent Recurrent Transformer (LRT), a lightweight augmentation of autoregressive transformers that reuses a high-level source-layer hidden state from the previous token as recurrent memory for the next token. Because this source state is already computed during ordinary decoding, LRT adds a cross-layer recurrent latent pathway across positions without inserting pause tokens or extra depth loops, and the standard attention mechanism and KV-cache interface are preserved.