AI RESEARCH
Ablating Archetypes: The Stability of Archetypal SAEs is an Artifact of Initialization and Metric Design
arXiv CS.LG
•
ArXi:2606.02061v1 Announce Type: new Dictionary learning with sparse autoencoders (SAEs) produces overcomplete bases from neural network activations that are often interpretable and reduces polysemanticity. However, features from SAEs vary substantially across random seeds -- a problem known as instability. Archetypal SAEs (Fel, 2025) were proposed as a general dictionary-learning intervention for reliable concept extraction, and report stable dictionaries at the end of