Integrating value functions and policy search for continuous Markov Decision Processes
Value function approaches for Markov decision processes ha ve been used successfully to find optimal policies for a large number of problems. Recent findi ngs have demonstrated that policy search can be used effectively in reinforcement learning wh en standard value function techniques become overwhelmed by the size and dimensionality of the state space. We demonstrate that substantial benefits can be achieved by combining the tw o approaches; we use an approximate value function solution in a low-dimensional space to s eed policy search in a continuous, high-dimensional space. We demonstrate our approach as a motion planner on a mobile ro ot. We show that this combination can in practice find good policies more efficiently t han policy search alone, and is capable of solving more complex problems than value functio s alone.