Aligned but Fragile: Enhancing LLM Safety Robustness via Zeroth-Order Optimization

ArXi:2605.29396v1 Announce Type: new Safety alignment for large language models (LLMs) aims to reduce harmful or unsafe behavior while preserving general utility. However, recent findings reveal that alignment effects can be fragile: lightweight post-alignment manipulations, such as parameter noise, activation noise, or quantization, can easily weaken the intended safety behavior.