Learning a Zeroth-Order Optimizer for Fine-Tuning LLMs

ArXi:2510.00419v2 Announce Type: replace Zeroth-order optimizers have recently emerged as an attractive approach for fine-tuning large language models (LLMs), as they avoid backpropagation and can substantially reduce memory overhead relative to standard first-order