AI RESEARCH
When LLM Reward Design Fails: Diagnostic-Driven Refinement for Sparse Structured RL
arXiv CS.LG
•
ArXi:2605.28918v1 Announce Type: new For sparse, structured reinforcement-learning tasks with semantic reward-function interfaces, LLM-generated reward shaping is better framed as debugging than one-shot generation. We study PPO-trained agents using MiniGrid as core evaluation and MuJoCo as boundary stress test. Our audit finds two dominant one-shot failure modes -- reward flooding and semantic/API misunderstanding -- plus a rarer weak-shaping case. We propose diagnostic-driven iterative refinement, where.