Reinforcement Learning and Its Relationship to Supervised Learning
The modern study of approximate dynamic programming (DP) combines ideas from several research traditions. Among these is the field of Artificial Intelligence, whose earliest period focussed on creating artificial learning systems. Today, Machine Learning is an active branch of Artificial Intelligence (although it includes researchers from many other disciplines as well) devoted to continuing the development of artificial learning systems. Some of the problems studied in Machine Learning concern stochastic sequential decision processes, and some approaches to solving them are based on DP. These problems and algorithms fall under the general heading of reinforcement learning. In this chapter, we discuss stochastic sequential decision processes from the perspective of Machine Learning, focussing on reinforcement learning and its relationship to the more commmonly studied supervised learning problems. Machine Learning is the study ofmethods for constructing and improving software systems by analyzing examples of their behavior rather than by directly programming them. Machine Learning methods are appropriate in application settings where people are unable to provide precise specifications for desired program behavior, but where examples of desired behavior are available, or where it is possible to assign a measure of goodness to examples of behavior. Such situations include optical character recognition, handwriting recognition, speech recognition, automated steering of automobiles, and robot control and navigation. A key property of tasks in which examples of desired behavior are available is that people can perform them quite easily, but people cannot articulate exactly how they perform them. Hence, people