AI RESEARCH
AdaptiveK: Complexity-Driven Sparse Autoencoders for Interpretable Language Model Representations
arXiv CS.LG
•
ArXi:2508.17320v3 Announce Type: replace Understanding the internal representations of large language models (LLMs) remains a central challenge for interpretability research. Sparse autoencoders (SAEs) offer a promising solution by decomposing activations into interpretable features, but existing approaches rely on fixed sparsity constraints that fail to account for input complexity. We propose AdaptiveK SAE (Adaptive Top K Sparse Autoencoders), a novel framework that dynamically adjusts sparsity levels based on the semantic complexity of each input.