Informedia-II : Auto-Summarization and Visualization Over Multiple Video
1 Overview The Informedia-II Project will change the paradigm for accessing digital video libraries through meaningful, manipulable overviews of video document sets, multimodal queries, and adaptive summarizations of very large amounts of video from heterogeneous distributed sources. Video information collages are the key technology in Informedia-II and will be built by advancing information visualization research to effectively deal with multiple video documents. A video information collage is a presentation of text, images, audio, and video derived from multiple video sources in order to summarize, provide context, and communicate aspects of the content for the originating set of sources. The collages to be investigated include chrono-collages emphasizing time, geo-collages emphasizing spatial relationships, and auto-documentaries which preserve video's temporal nature. Users will be able to interact with the video collages to generate multimodal queries across time, space, and sources. Video collages are made adaptive by giving preference to the concepts and query terms in the user's interaction history. The synthesis and summarization functions underlying these collages will be made possible through extensions of text clustering and expectation maximization algorithms to video and audio features. 2.1 Video Information Collages: Adaptive Visualization and Summarization 2.1.1 Interactive visualization of digital library data and metadata Much current research on digital libraries focuses on named entity extraction and transformation into structured metadata. Examples include entities like events, people, and places, and attributes like birth date or latitude. Unfortunately, this extraction process is not very reliable, and in any case a digital library may contain references to hundreds of thousands of entities. We have been integrating Visage and Visage functionality for visualization and semantic zooming within the Informedia library. Information visualization is a powerful tool for summarizing large sets of structured data about entities and their relationships to uncover overall patterns, and then drilling down into interesting subsets. We have applied this technique to metadata extracted from the Informedia Digital Video Library, and demonstrated examples of conclusions that can be drawn from metadata patterns alone at JCDL'01. Currently, text attributes are handled poorly in terms of both query semantics and interaction speed. Our goal is to overcome these difficulties, so that integrated data and metadata library browsing becomes a continuous interactive activity. The visual query language allows database-style joins between entities of different types, Dynamic Query filtering of attribute values, visualization of conditional attribute value distributions with histograms, and drill down to individual entities [Derthick1997]. For instance, the …