A Performance Evaluation Protocol for Content-Based Image Retrieval Algorithms/Systems
In this paper, we address an important yet controversial issue in content-based image retrieval (CBIR): performance evaluation (PE). The goal is to develop an objective and comprehensive protocol for quantitative evaluation and comparison of CBIR performances among different approaches. The protocol consists of two elements: acquisition of appropriate ground truth data, and definition of quantitative performance metrics. A large image data set with human-label ground truths has been built as part of the protocol and made available for public use. An application of this protocol to our MiAlbum system is also presented, which shows suitability of such a protocol.