Video Browsing Using Cooperative Visual and Linguistic Indices

摘要

Management of video information for content-based retrieval has been recognizedas a significant problem in the multimedia community for a long time. Recent literature reports research activities in three major areas: segmentation of special video events based on both abrupt and gradual scene changes; motion analysis techniques applied toward determination of camera movement and objectbackground segmentation; and data-modeling and query-language development with the assumption that objects in the video have been segmented by low-level processing. The proposal was chartered to explore methods and develop techniques for information retrieval and browsing of news video using both imagery and linguistic content. We chose television news as the domain of this video retrieval project. The long-term goal for video news information retrieval/browsing system is to have the ability to: (1) isolate a specific information item using imagery, linguistic, and audio description and any combination of these; (2) present it at different levels of abstraction (detail) using any a priori knowledge about the domain of application to the maximum extent possible; (3) given one piece of information related to a semantic information unit such as a news story, find other parts of the same or related semantic units; (4) allow a user to put in further annotations as additional information related to a semantic information unit like a news item, and (5) allow the user to search through the collection of news videos using any prior annotation that the user has made. Fulfilling the contract required us to develop a prototype Video Data Management System, which demonstrated the infrastructural framework for exploring the research issues listed in the proposal. The system has both hardware and software components which together allow the complete management of video data.