Learning Hierarchical Behaviors
In the last few years, many researchers have beg un to study how to introduce hierarchy into reinforcement learning methods. Generally, th is work has pushed the envelope in one of several important directions, but it typically has voided the question of how to learn systems of hierarchical behavior from experience. We present a preliminary system for learning simple hierarchical systems using a constrained language f or behavior specification. The system learns “do-until” macros by choosing a subgoal from and us ing its past experience with respect to this subgoal to build a constrained macro behavior that achieves it. The system can either be run offline and used for new but similar problems, or c an be used during learning. We present an algorithm that combines macro learning with asynchr onous value iteration methods such as prioritized sweeping. Macros learned by the system for a simple environment are discussed. Additionally, we present the related literature and sketch out the future work for this project.