The role of measurement reliability in clinical trials

Abstract

One of the principal characteristics of an outcome measure in a clinical trial, and any measurement in general, is its reliability. Reliability refers to the reproducibility of the measurement when repeated at random in the same subject or specimen. Reliability is often confused with validity, which refers to the extent to which the variable properly measures the underlying trait of interest. The coefficient of reliability is an estimate of the proportion of all variation that is not due to measurement error and is readily estimated from replicate measurements. The reliability of a measurement determines its maximal correlation or R2 and slope (or effect size) in regression models, its sensitivity and specificity when used for classifications or predictions, and the power of a statistical test employing the measurement. All decline as the reliability of the measure declines. The reliability of a measurement is an important consideration in the choice of the primary outcome measure for a clinical trial and in the choice of measures used for assessment of eligibility and exclusion. Reliability of measures should be assessed and assured by a quality control program based on randomly selected duplicate assessments. Just as the power of a study is reported in a final publication, so also should the reliability of the outcome and eligibility measurements so as to allow the authors to better describe, and readers to better understand, the sources of imprecision in study results, and those who follow to improve the design of future trials.