Discovering objects and their location in images with Latent Dirichlet Allocation
We seek to discover object categories and their locations in a set of unlabelled images. We achieve this using probabilistic models developed in the text understanding community to discover interesting topics in a corpus of text documents. We hope that the application of these models to a set of images will discover visual topics corresponding to object categories. We show how to form the visual analogue of text documents by vector quantizing SIFT-like regions at points found through two types of interest point detectors. We primarily focus on the Latent Dirichlet Allocation (LDA) model and compare with the very similar probabilistic Latent Semantic Analysis (pLSA) model. We describe the LDA model and the Gibbs sampling solution for Bayesian inference. We fit the LDA model to image sets for which we know ground-truth labels and investigate the visual topics that are discovered. In addition, for images with primarily a single object category depicted, we use the inferred LDA parameters to assign these images to the discovered visual topics and evaluate the goodness of these assignments through confusion matrices. Furthermore, we evaluate assignments to topics on unseen images and compare with an existing supervised method. Finally, we formulate a method for localizing objects in an image and evaluate the resulting segmentations. Since the LDA model assumes that words are exchangable in a document, we show how to improve the segmentations by introducing a new vocabulary called ‘doublets’, which encode spatially local co-occurring regions.