AI RESEARCH
The Implicit Bias of Depth: From Neural Collapse to Softmax Codes
arXiv CS.LG
•
ArXi:2605.23087v1 Announce Type: new Neural collapse (NC) describes the structured geometry that emerges in the features and weights of trained classifiers. Recent theory suggests NC can be suboptimal in deep architectures, attributing this to an explicit low-rank bias from L2 regularization. We study the deep unconstrained feature model (UFM)-equivalent to a deep linear network with orthogonal inputs-trained without regularization, to isolate how gradient descent and depth alone shape NC.