AI RESEARCH
Beyond Routing: Characterising Expert Tuning and Representation in Vision Mixture-of-Experts
arXiv CS.CV
•
ArXi:2605.20610v1 Announce Type: new Mixture-of-Experts (MoE) models are often interpreted by analysing which categories are routed to which experts. However, routing alone does not reveal what each expert actually encodes. We train sparsely-gated convolutional MoE models with a contrastive objective on natural images and characterise expert specialisation using tools from visual neuroscience. Extending from gating-level to expert-level analyses, we measure per-expert category separability, and per-expert tuning using the most exciting inputs.