A review of intelligent content-based indexing and browsing of medical images

Abstract

Physicians are beginning to be able to gain access, through the Internet, to the world’s collections of multimedia medical information such as MRI (magnetic resonance imaging) and CT (computer tomography) image archives, videos of surgical operations and medical lectures, textual patient records and media annotations. New techniques and tools are needed to represent, index, store and retrieve digital content efficiently across large collections. In this review, we trace the development of visual information systems for healthcare and medicine from Picture Archiving and Communications Systems (PACS) to the recent advances in content-based image retrieval, whereby images are retrieved based on their visual content similarity - that is, colour, texture, and shape. Medical images, unlike consumer-oriented images, pose additional challenges to content-based image retrieval, in that visual features of normal and pathological images are typically separated by only subtle differences in visual appearance. Intelligent image retrieval and browsing therefore requires a combination of prior knowledge of the medical domain, image content and image annotation analysis. To this end, we also overview the I-Browseproject, conducted jointly by the Clinical School of the University of Cambridge and the City University of Hong Kong, which aims to develop techniques which enable a physician to search over image archives through a combination of semantic and iconic contents.