Model-Based Reinforcement Learning with an Approximate, Learned Model
Model-based reinforcement learning, in which a model of the environment's dynamics is learned and used to supplement direct learning from experience, has been proposed as a general approach to learning and planning. We present the rst experiments with this idea in which the model of the environment's dynamics is both approximate and learned online. These experiments involve the Mountain Car task, which requires approximation of both value function and model because it has continuous state variables. We used models of the simplest possible form, state-aggregation or \grid" models, and CMACs to represent the value function. We nd that model-based methods do indeed perform better than model-free reinforcement learning.