Bellman Residual Minimization for Control: Geometry

Bellman Residual Minimization for Control: Geometry, Stationarity, and Convergence

arXiv:2601.18840v4 Announce Type: replace Abstract: Markov decision problems are most commonly solved via dynamic programming. Another approach is Bellman residual minimization, which directly minimizes the squared Bellman residual objective function. However, compared to dynamic programming, this approach has received relatively less attention, mainly because it is often less efficient in practice and can be more difficult to extend to model-free settings such as reinforcement learning.