Perceptually Organized Em: a Framework for Motion Segmentation That Combines Information about Form and Motion
Recent progress in motion analysis has been achieved with systems that estimate global pa-rameterized motion by integrating multiple constraints. The success of these approaches depends critically on the ability to segment constraints derived from diierent motions. Hence the problems of motion estimation and segmen-tation are tightly coupled. We believe it is impossible to solve these problems solely in the motion domain, and that mechanisms of spatial form analysis must be incorporated into the motion estimation procedure. We present a new framework which allows the incorporation of form information in a graceful manner. It combines concepts from perceptual organization with the powerful optimization technique of EM. We show that the algorithm is guaranteed to decrease a cost function at every iteration, and that in the absence of form information the cost function reduces to the one minimized by EM. We demonstrate that the approach can achieve good motion estimation and segmentation with challenging motion sequences. Recent progress in motion analysis has been achieved with systems that estimate global parameterized mo-These methods have advantages over local optic ow in that they overcome the local ill-posedness of the motion estimation problem by integrating multiple constraints. The sucess of these approaches , however, depends critically on the ability to segment constraints derived from diierent motions. Hence the problems of motion estimation and segmentation have become tightly coupled. The joint solution of these problems remains diicult, even for scenes that are very simple. Consider, for example, the scene shown in g 1(a) (see also Bergen et al., 1990]). Two bars of diierent grey shades are moving, one to the left and one to the right. We will consider how several kinds of motion analyses treat this input. First, the output of a standard least-squares optic ow routine is shown in g. 1(b), as an arrow plot; the x and y components of velocity are shown in g. 1(c) and (d) (velocities below some threshold conndence are set to zero, a b c d Figure 1: a A simple image sequence which causes problems for traditional motion estimation algorithms. b Least squares optical ow shown as an arrow plot c Least squares optical ow horizontal component. d Least squares optical ow vertical component. the algorithm is an implementation of Lucas and Kanade (1981) modiied according to Simoncelli et al., 1991]). Although this sequence is a synthetic one, it illustrates problems that occur frequently in analyzing real …