Reinforcement Learning with Stability Guarantees TITLE2
We present a new approach to improving the reliability and overall performance of reinforcement learning (RL) control for minimum cost-to-target problems. In particular, we propose combining RL with Lyapunov-based control, an analytical approach from control theory for designing controllers that make dynamical systems stable. By building Lyapunov-based constraints into a control architecture, we can prove that the controlled system will be brought to a desired target state. At the same time, RL is used to minimize the cost incurred by the controller. The resulting controllers combine theoretical guarantees on their behavior with improved practical performance compared to either standard RL or Lyapunov-based methods alone. Because the controllers ensure stability by design, the guarantees we establish hold during learning as well as after. In fact, the guarantees hold independently of many characteristics of the RL method being used, including the method of function approximation. We illustrate our techniques on a pendulum swing-up task.