Reinforcement Learning in Large, High‐Dimensional State Spaces
The previous chapter addresses the ?>curse of dimensionality ?> by treating ADP as the dual of the linear programming problem and introduces the method known as approximate linear programming. This chapter presents another method for dealing with the "curse of dimensionality," the policy gradient reinforcement learning framework. The Action Transition Policy Gradient (ATPG) algorithm presented here estimates a gradient in the policy space that increases reward. Following a brief motivation the authors present their algorithm in detail and discuss its properties. Finally, detailed experimental results are presented to show the types of problems that the algorithm can be applied to and what type of performance can be expected. Another algorithm, Boundary Localized Reinforcement Learning, is also discussed in this chapter. This is a mode switching controller that can be used to increase the rate of convergence.