A Critique of Methods in Reported Studies of Long-term Vascular Complications in Patients with Diabetes Mellitus

Abstract

From 149 papers that referred to the development of vascular complications in patients with diabetes mellitus, we selected thirty-three that could be used for analysis of methodology. Eight of these reports contained inception cohorts in which a valid statistical risk could be calculated from the target events (cited in the numerators) and the initial states (cited in the denominators). Twenty-five reports contained longitudinal prevalence groups that were suitable only for appraisal of the target events cited in the numerators. The denominator populations in the cohort studies were usually reported without reproducible specifications of the proportionate sources of the cohort, the criteria for diagnosing diabetes mellitus, and the criteria for excluding patients from the investigation. Although most of the starting patients were “recovered” in reports of final results, the denominators of different results were often unstable, reflecting an absence of suitable initial data. The initial state of the cohorts was usually reported for age and sex, less commonly for metabolic features of the diabetes, and rarely for comorbid diseases that were unrelated to diabetes. The initial-state retinopathy, nephropathy, hypertension, peripheral vascular and cardiac ailments that were “related” to diabetes were reported infrequently. When reported, these conditions were not always specified with reproducible diagnostic criteria or cited quantitatively for prevalence in the cohort. The incidence of subsequent vascular complications was usually reported without satisfactory distinctions between the occurrence of a new complication and a change in the state of a previous one. The complications were generally cited with “objective” data that did not indicate the concomitant symptomatic and other functional effects. Few of the cohort investigators took appropriate advantage of the opportunity to correlate subsequent complications with different “risk factors” of the initial state. In longitudinal prevalence studies, the complications were correlated mainly with duration and regulation of the diabetes, but seldom with the initial distinctions of the patients. Since these problems arise from defects in selecting, specifying, and analyzing the original and subsequent clinical condition of the treated patients, and since the randomized assignment of treatment does not deal with these problems, a predominantly statistical approach to therapeutic trials cannot be expected to resolve the existing scientific controversies about treatment.