PAC-Bayesian Reinforcement Learning Trains Generalizable Policies

ArXi:2510.10544v3 Announce Type: replace-cross We derive a novel PAC-Bayesian generalization bound for reinforcement learning that explicitly accounts for Marko dependencies in the data, through the chain's mixing time. This contributes to overcoming challenges in obtaining generalization guarantees for reinforcement learning, where the sequential nature of data breaks the independence assumptions underlying classical bounds. The new bound provides non-vacuous certificates for modern off-policy algorithms such as Soft Actor-Critic.