An Inventory of Evaluation Studies of Information Technology in Health Care

Abstract

Summary Objectives: During the last years the significance of evaluation studies as well as the interest in adequate methods and approaches for evaluation has grown in medical informatics. In order to put this discussion into historical perspective of evaluation research, we conducted a systematic review on trends in evaluation research of information technology in health care from 1982 to 2002. Methods: The inventory is based on a systematic literature search in PubMed. Abstracts were included when they described an evaluation study of a computer-based component in health care. We identified 1035 papers from 1982 to 2002 and indexed them based on a multi-axial classification describing type of information system, study location, evaluation strategy, evaluation methods, evaluation setting, and evaluation focus. Results and Conclusions: We found interesting developments in evaluation research in the last 20 years. For example, there has been a strong shift from medical journals to medical informatics journals. With regard to methods, explanatory research and quantitative methods have dominated evaluation studies in the last 20 years. Since 1982, the number of lab studies and technical evaluation studies has declined, while the number of studies focusing on the influence of information technology on quality of care processes or outcome of patient care has increased. We interpret this shift as a sign of maturation of evaluation research in medical informatics.