Semantic Multimedia Content Retrieval and Filtering

Abstract

Several consumer electronic devices that allow capturing digital multimedia content (like mp3 recorders, digital cameras, DVD camcorders, smart phones etc.) are available today. These devices have allowed both the amateur and the professional users to produce large volumes of digital multimedia material, which, together with the traditional media objects digitized recently (using scanners, audio and video digitization devices) form a huge distributed multimedia information source. The multimedia material that is available today is usually organized in independent multimedia information sources, developed on top of different software platforms. The Internet, the emergence of advanced network infrastructures that allow for the fast, efficient and reliable transmission of multimedia content and the development of digital multimedia content services on top of them form an open multimedia consumption environment. In this environment, the users access the multimedia material either through computers or through cheap consumer electronic devices that allow the consumption and management of multimedia content. The users of such an open environment need to be able to access the services offered by the different vendors in a transparent way and to be able to compose the different atomic services (like, for example, multimedia content filtering) into new, composite ones. In order to fulfill this requirement, interoperability between the multimedia content services offered is necessary. Interoperability is achieved, at the syntactic level, through the adoption of standards. At the semantic level, interoperability is achieved through the integration of domain knowledge expressed in the form of domain ontologies. An ontology is a logical theory accounting for the intended meaning of a formal vocabulary, i.e. its ontological commitment to a particular conceptualization of the world (Guarino, 1998). The standard that dominates in multimedia content description is the MPEG-7 (Salembier, 2001), formally known as Multimedia Content Description Interface. It supports multimedia content description from several points of view, including media information, creation information, structure, usage information, textual annotations, media semantics, and low-level visual and audio features. Since the MPEG-7 allows the structured description of the multimedia content semantics, rich and accurate semantic descriptions can be created and powerful semantic retrieval and filtering services can be built on top of them. It has been shown, in our previous research (Tsinaraki, Fatourou and Christodoulakis, 2003), that domain ontologies capturing domain knowledge can be expressed using pure MPEG-7 constructs. This way, domain knowledge can be integrated in the MPEG-7 semantic descriptions. The domain knowledge is subsequently utilized for supporting semantic personalization, retrieval and filtering and has been shown to enhance the retrieval precision (Tsinaraki, Polydoros and Christodoulakis, 2007). Although multimedia content description is now standardized through the adoption of the MPEG-7 and semantic multimedia content annotation is possible, multimedia content retrieval and filtering (especially semantic multimedia content retrieval and filtering), which form the basis of the multimedia content services, are far from being successfully standardized.