MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Any-Precision LLM

ArXi:2602.20191v2 Announce Type: replace-cross Dynamic runtime latency and memory constraints necessitate flexible large language model (LLM) deployment, where an LLM can be inferred with various quantization precisions based on available computational resources. Recent work on such any-precision quantization either relies on hardware-inefficient vector quantization or induces additional scaling factors when switching between bit-widths. Meanwhile, existing post-