Foreground/background Separation Via Camera Motion Estimation
Camera motion estimation is an important step in the compression of video sequences taken with a real camera. By obtaining zoom and pan parameters that account for camera translation, rotation and change of focus, it is possible to devote all the subsequent compression efforts on the coding of the foreground motion. In this paper, we propose a new approach to zoom and pan detection. First, the images are processed by an edge detection algorithm. Then, features are matched in subsequent frames of the sequence and zoom and pan parameters are obtained by minimizing the error between the matches found and the matches that would have been found in presence of only zoom and pan. This approach offers several advantages. First, on a serial machine such as Sparc I1 workstation, it is an order of magnitude faster than zoom and pan detection algorithms based on traditional Block Matching Algorithms. Second, the edge image and the matching coordinates can be used to code the movement other than zoom and pan. Third, the minimum square error approach is very convenient computationally and guarantees the best prediction of the parameters. Once zoom and pan parameters are determined, the frame difference between the actual next frame and its prediction based on zoom and pan parameters essentially contains foreground motion. To examine the effectiveness of our foreground/background separation technique, we apply our algorithm to two sequences: a high resolution "MIT" sequence where only zoom and pan are present, and a "ping-pong" sequence where foreground motion is also present. Our data also shows that zoom and pan parameters can be estimated considerably more accurately, if the motion vectors are determined to sub pixel accuracy. This is achieved by interpolating the images before passing them through the edge detector.