Single Timescale Actor-Critic Method to Solve the Linear Quadratic Regulator with Convergence Guarantees
Mo Zhou, Jianfeng Lu; 24(222):1−34, 2023.
Abstract
We propose a single timescale actor-critic algorithm to solve the linear quadratic regulator (LQR) problem. A least squares temporal difference (LSTD) method is applied to the critic and a natural policy gradient method is used for the actor. We give a proof of convergence with sample complexity O(ε−1log(ε−1)2). The method in the proof is applicable to general single timescale bilevel optimization problems. We also numerically validate our theoretical results on the convergence.
© JMLR 2023. (edit, beta) |