Enhancing Layer Attention Efficiency through Pruning Redundant Retrievals

ArXi:2503.06473v5 Announce Type: replace-cross Growing evidence suggests that layer attention mechanisms, which enhance interaction among layers in deep neural networks, have significantly advanced network architectures. However, existing layer attention methods suffer from redundancy, as attention weights learned by adjacent layers often become highly similar. This redundancy causes multiple layers to extract nearly identical features, reducing the model's representational capacity and increasing