Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories

ArXi:2606.03979v1 Announce Type: new The past few decades have witnessed significant advances in the design of machine learning algorithms, from early studies on task-specific shallow models to general deep Large Language Models (LLMs). Despite showing promising results in tasks that require instant prediction or in-context learning, existing models lack the ability to continually learn and effectively transfer their temporal in-context knowledge to their long-term parameters. Inspired by human learning process, we.