Decision-theoretic Image Retrieval with Embedded Multi-resolution Mixtures
The design of an effective architecture for image retrieval requires careful consideration of the interplay between the three major components of a retrieval system: feature transformation, feature representation, and similarity function. We introduce a decision theoretic formulation of the retrieval problem that enables the design of systems where all components are optimized with respect to the same end-to-end performance criteria: the minimization of the probability of retrieval error. The new formulation is shown to have two appealing properties. First, it leads to an optimal similarity function (the posterior probability of the query under the database image class) that generalizes many of its previously proposed counterparts. Second, it enables a theoretical characterization of the impact of the feature transformation and representation in the probability of error. In addition to exposing the major limitations of a large body of previous retrieval approaches, this characterization allows the derivation of a series of conditions for the optimal design of the feature transformation and representation. The search for a practical solution that can satisfy these conditions leads to the adoption of an embedded multi-resolution mixture representation and originates an efficient algorithm for optimal feature selection. The resulting retrieval architecture achieves a good compromise between retrieval accuracy, invariance, perceptual relevance of similarity judgments, and complexity. Extensive experimental results show that decision-theoretic retrieval performs well on color, texture, and generic image databases in terms of both retrieval accuracy and perceptual relevance of similarity judgments. Author email: nuno.vasconcelos@compaq.com c Compaq Computer Corporation, 2002 This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of the Cambridge Research Laboratory of Compaq Computer Corporation in Cambridge, Massachusetts; an acknowledgment of the authors and individual contributors to the work; and all applicable portions of the copyright notice. Copying, reproducing, or republishing for any other purpose shall require a license with payment of fee to the Cambridge Research Laboratory. All rights reserved. CRL Technical reports are available on the CRL’s web page at http://crl.research.compaq.com. Compaq Computer Corporation Cambridge Research Laboratory One Cambridge Center Cambridge, Massachusetts 02142 USA