AI RESEARCH
L$^3$: Large Lookup Layers
arXiv CS.AI
•
ArXi:2601.21461v3 Announce Type: replace-cross Modern sparse language models typically achieve sparsity through Mixture-of-Experts (MoE) layers, which dynamically route tokens to dense MLP "experts." However, dynamic hard routing has a number of drawbacks, such as potentially poor hardware efficiency and needing auxiliary losses for stable