Correcting Gradient-Based Circuit Localization via Interaction-Aware Backpropagation

ArXi:2505.17630v4 Announce Type: replace-cross Circuit localization methods aim to identify the subset of model components responsible for specific behaviors in large language models, enabling detailed mechanistic analysis. Most existing methods assume components act independently and estimate importance by perturbing each component in isolation. However, components in neural networks interact, and ignoring these interactions leads to systematic misestimation of component importance.