HEAPr: Hessian-based Efficient Atomic Expert Pruning in Output Space

ArXi:2509.22299v3 Announce Type: replace-cross Mixture-of-Experts (MoE) architectures in large language models (LLMs) deliver exceptional performance and reduced inference costs compared to dense LLMs. However, their large parameter counts result in prohibitive memory requirements, limiting practical deployment. While existing pruning methods primarily focus on expert-level pruning, this coarse granularity often leads to substantial accuracy degradation. In this work, we