AI RESEARCH
From Correlation to Cause: A Five-Stage Methodology for Feature Analysis in Transformer Language Models
arXiv CS.CL
•
ArXi:2605.22462v1 Announce Type: new We propose a five-stage methodology for causal feature analysis in transformer language models (probe design, feature extraction, causal validation, robustness testing, and deployment integration) and nstrate it end-to-end on GPT-2 small performing the Indirect Object Identification (IOI) task. Activation patching recovers the canonical IOI circuit (layer-9 head 9 alone gives recovery +1.02). A sparse autoencoder recovers per-name selective features with effect sizes of 30 to 50 activation units.