Framework for Integration of Medical Image and Text-Based Report Retrieval to Support Radiological Diagnosis

Abstract

Medical images are vital part of diagnostics and patient treatment. With the advent of technology, there is a rapid increase in the number of radiological images produced every day. Attempts have been made to use a Content Based Image Retrieval (CBIR) approach for assisting in radiological diagnosis. However, this approach suffers from the semantic gap problem. Few text retrieval systems are in place for assisting the radiologist to retrieve similar past cases. However, for the least experienced radiologist it is hard to describe the unknown case using text query. Therefore, the aim of this chapter is integrating the radiological CBIR and text based reports retrieval in order to support radiological diagnosis. The proposed technique is described in three stages: a) retrieval by image similarity, b) retrieval by text, and c) fusion of image and text retrieval for better diagnosis. Number of experiments are demonstrated along with their evaluation techniques on mammogram image database.