Spoken Dialogue Management Using Hierarchical Reinforcement Learning and Dialogue Simulation
Speech-based human-computer interaction faces several difficult challenges in order to be more widely accepted. One of the challenges in spoken dialogue management is to control the dialogue flow (dialogue strategy) in an efficient and natural way. Dialogue strategies designed by humans are prone to errors, labour-intensive and non-portable, making automatic design an attractive alternative. Previous work proposed addressing the dialogue strategy design as an optimization problem using the reinforcement learning framework. However, the size of the state space grows exponentially according to the state variables taken into account, making the task of learning dialogue strategies for large-scale spoken dialogue systems difficult. In addition, learning dialogue strategies from real users is a very expensive and time-consuming process, making automatic learning an attractive alternative. To address these research problems three lines of investigation are proposed. Firstly, to investigate a method to simulate task-oriented human-computer dialogues at the intention level in order to design the dialogue strategy automatically. Secondly, to investigate a metric to evaluate the realism of simulated dialogues. Thirdly, to make a comparative study between hierarchical reinforcement learning methods and reinforcement learning with function approximation, in order to find an effective and efficient method to learn optimal dialogue strategies in large state spaces. Finally, a timeline for the completion of this research is proposed.