Bias in vaccine effectiveness studies of clinically severe outcomes that are measured with low specificity: the example of COVID-19-related hospitalisation

Abstract

Many vaccine effectiveness (VE) analyses of severe disease outcomes such as hospitalisation and death include ‘false’ cases that are not actually caused by the infection or disease under study. While the inclusion of such false cases inflate outcome rates in both vaccinated and unvaccinated populations, it is less obvious how they affect estimates of VE. Illustrating the main points through simple examples, this article shows how VE is underestimated when false cases are included as outcomes. Depending how the outcome indicator is defined, estimates of VE against severe disease outcomes, whose definition allows for the inclusion of false cases, will be biased downwards and may in certain circumstances approximate the same level as the VE against infection. The bias is particularly pronounced for vaccines that offer high levels of protection against severe disease outcomes but poor protection against infection. Analysing outcomes that are measured with low sensitivity generally does not cause bias in VE studies; defining outcome indicators that minimise the number of false cases rather than the number of missed cases is preferable in VE studies.