INTERACTION BETWEEN OBJECTIVE PERFORMANCE MEASURES AND SUBJECTIVE USER PERCEPTIONS IN THE EVALUATION OF MEDICAL DEVICES: A CASE STUDY

Abstract

Objectives: Medical technology is a large and expanding industry. Introducing new medical devices is important but several challenges exist in implementing the optimal method of evaluation. Both objective and subjective measures can be used for evaluation. The former is the mainstay of evaluation, yet subjective assessment is often the basis for the introduction of new medical technology. The aim of this study was to determine the interaction and concordance between objective and subjective assessment of new medical technology.Methods: This study used both objective performance measures and subjective user perceptions in the evaluation of a new medical device designed to improve the accuracy of gravity-assisted delivery of intravenous fluids, compared with the current, widely used “roller-clamp” device. The concordance of objective and subjective assessments was evaluated using comparative analysis.Results: Objective assessment of the accuracy of intravenous fluid delivery revealed no difference between the two devices (p = .636). Subjective assessment revealed that the new device was perceived to be significantly more accurate (p = .001). This lack of concordance can be partially explained by both device and demand characteristics.Conclusions: This case study reveals a significant discordance between the objective and subjective assessments. It provides some explanation for why new medical devices are adopted without objective evidence of benefit. This phenomenon has been termed “persuasive design” and its influence should be controlled for in the evaluation, purchase and introduction of new medical devices. This should help reduce the risk and associated cost of premature introduction.