AI RESEARCH
When Data Is Scarce: Scaling Sparse Language Models with Repeated Training
arXiv CS.AI
•
ArXi:2606.01155v1 Announce Type: cross Scaling laws for dense LLMs under infinite data are well explored, but how sparsity interacts with limited data is not. In this work, we study sparse