AI RESEARCH
Target Updates May Stabilize Linear Q-Learning: Periodic and Soft Dynamics
arXiv CS.LG
•
ArXi:2606.02645v1 Announce Type: cross Periodic target updates in Q-learning and soft target updates in actor-critic methods are empirically well established stabilization mechanisms, but their precise theoretical explanation is still incomplete. This paper gives a rigorous and exact analysis of these mechanisms for Q-learning with linear function approximation (linear Q-learning) using the exact switched linear system (SLS) dynamics induced by the Bellman maximum and the joint spectral radius (JSR) of the resulting switching matrix families.