AI RESEARCH
$A^2$: Smaller Self-Supervised ViTs Localize Better than Larger Ones
arXiv CS.CV
•
ArXi:2606.03148v1 Announce Type: new Robust visual classification often depends on localizing the main foreground objects in an image while ignoring contextual distractors. Surprisingly, we find that the attention maps of smaller self-supervised ViTs localize foreground objects better than those of larger ViTs. However, we still need large ViTs, because they extract richer representations from each patch.