Law of Neural Interaction: Depth-Width Shape, Interaction Efficiency, and Generalization

ArXi:2605.27989v1 Announce Type: new The guidance of scaling laws has increased the resource demands of modern large language models (LLMs), yet it remains questionable whether these models utilize resources effectively under a fixed budget. Previous research has proved superposition as a key contributor to loss. By leveraging the Neural Feature Ansatz, we extend superposition from parameter space to gradient space and define it as neural interaction.