Beware of on-treatment safety analyses

Abstract

Introduction: Assessing safety is important to evaluating new medications. In many randomized clinical trials, assessment of safety relies on so-called on-treatment analysis, where data on adverse events are collected only while the participant is taking study medication and perhaps for a few (7, 14, or 30) days after stopping. This article discusses the consequence of such failure to use intent-to-treat analyses in assessing safety. Methods: This article discusses two approaches to analysis of safety data: intention-to-treat and on-treatment analysis with reference to principles of the design of randomized clinical trial. Results: On-treatment analysis violates randomization and is often not well defined. Moreover, because the typical on-treatment analysis ignores the reason participants in clinical trials stop treatment, on-treatment analyses can lead to biased estimates of risk. Examples show biases that can result from failure to count all adverse events. An example from a study of rofecoxib shows an on-treatment analysis that led to likely underestimation of harm; an example from a study of saxagliptin shows an on-treatment analysis that led to a likely overestimate of harms. Conclusion: For major safety outcomes in long-term clinical trials, intention-to-treat analysis should be performed in the framework of benefit–risk evaluation. More generally, analyses of safety should be tailored to the specific question being asked with the specific study design under consideration. On-treatment analyses are subject to bias; however, the direction of that bias is not necessarily clear.