Computer Vision for Human–Machine Interaction: Probabilistic Models of Verbal and Body Gestures
This chapter describes several probabilistic techniques for representing, recognizing, and generating spatiotemporal connguration sequences. We rst describe how such techniques can be used to visually track and recognize lip movements to augment a speech recognition system. We then demonstrate additional techniques that can be used to animate video footage of talking faces and synchronize it to diierent sentences of an audio track. Finally we outline alternative low-level representations that are needed to apply these techniques to articulated body gestures. 7.1 Introduction Gestures can be described as characteristic conngurations over time. While uttering a sentence we express very ne grained verbal gestures as complex lip conngurations over time, and while performing body actions, we generate articulated connguration sequences of jointed arm and leg segments. Such conngurations lie in constrained subspaces and diierent y Part of this report is published at ICCV'95 9]