Using Information Retrieval for Interaction with Mobile Devices

Abstract

Future's environments will be sensitive and responsive to the presence of people to support them carrying out their everyday life activities, tasks and rituals, in an easy and natural way. Such interactive spaces will use the information and communication technologies to bring the computation into the physical world in order to enhance ordinary activities of their users. Human-computer interaction (HCI) and information retrieval (IR) fields have both developed innovative techniques to address the challenge of navigating complex information spaces, but their insights have often failed to cross-disciplinary borders. Human-computer information retrieval (HCIR) has emerged in academic research and industry practice to bring together research in the fields of IR and HCI, in order to create new kinds of search systems that depend on continuous human control of the search process. HCIR is the study of information retrieval techniques that bring human intelligence into the search process. This chapter will describe search-based interaction techniques using two human-computer interaction information retrieval systems: (1) a speech-based spoken multimedia retrieval system that can be used to present relevant video-podcast (vodcast) footage in response to spontaneous speech and conversations during daily life activities, and (2) a novel shape retrieval technique that allows 3D modeling of indoor/outdoor environments using multi-view sketch input from a mobile device.