Adaptive designs in multi-reader multi-case clinical trials of imaging devices

Abstract

Evaluation of medical imaging devices often involves clinical studies where multiple readers (MR) read images of multiple cases (MC) for a clinical task, which are often called MRMC studies. In addition to sizing patient cases as is required in most clinical trials, MRMC studies also require sizing readers, since both readers and cases contribute to the uncertainty of the estimated diagnostic performance, which is often measured by the area under the ROC curve (AUC). Due to limited prior information, sizing of such a study is often unreliable. It is desired to adaptively resize the study toward a target power after an interim analysis. Although adaptive methods are available in clinical trials where only the patient sample is sized, such methodologies have not been established for MRMC studies. The challenge lies in the fact that there is a correlation structure in MRMC data and the sizing involves both readers and cases. We develop adaptive MRMC design methodologies to enable study resizing. In particular, we resize the study and adjust the critical value for hypothesis testing simultaneously after an interim analysis to achieve a target power and control the type I error rate in comparing AUCs of two modalities. Analytical results have been derived. Simulations show that the type I error rate is controlled close to the nominal level and the power is adjusted toward the target value under a variety of simulation conditions. We demonstrate the use of our methods in a real-world application comparing two imaging modalities for breast cancer detection.