Use of causal claims in observational studies: a research on research study

Abstract

AbstractObjectiveTo evaluate the consistency of causal statements in the abstracts of observational studies published in The BMJ.DesignResearch on research study.Data sourceAll cohort or longitudinal studies describing an exposure-outcome relationship published in The BMJ during 2018. We also had access to the submitted papers and reviewer reports.Main outcome measures:Proportion of published research papers with ‘inconsistent’ use of causal language in the abstract. Papers where language was consistently causal or non-causal were classified as ‘consistently causal’ or ‘consistently not causal’, respectively; those where causality may be inferred were classified as ‘suggests causal’. For the ‘inconsistent’ papers, we then compared the published and submitted version.ResultsOf 151 published research papers, 60 described eligible studies. Of these 60, we classified the causal language used as ‘consistently causal’ (13%), ‘suggests causal’ (35%), ‘inconsistent’ (20%) and ‘consistently not causal’(32%). The majority of the ‘Inconsistent’ papers (92%) were already inconsistent on submission. The inconsistencies found in both submitted and published versions was mainly due to mismatches between objectives and conclusions. One section might be carefully phrased in terms of association while the other presented causal language. When identifying only an association, some authors jumped to recommending acting on the findings as if motivated by the evidence presented.ConclusionFurther guidance is necessary for authors on what constitutes a causal statement and how to justify or discuss assumptions involved. Based on screening these abstracts, we provide a list of expressions beyond the obvious ‘cause’ word which may inspire a useful more comprehensive compendium on causal language.Strengths and limitations of this studyWe present examples of ambiguous causal statements in published abstracts of observational studies in a high impact journalWe focused on the abstract where clear messages are especially important, as many readers just read the abstract of a studyThe focus on the abstract may miss further discussion on the validity of underlying assumptions justifying causal inference in the setting studied.The prevalence and nature of the problems found is a call for better instruction on and consideration of causal language throughout the editorial process in clinical and epidemiological research.We provide a list of words and study elements that could point in the direction of causality or otherwise, which may inspire a more comprehensive compendium.