To share or not to share data: how valid are trials evaluating first-line ovulation induction for polycystic ovary syndrome?

Abstract

Abstract BACKGROUND In our recent individual participant data (IPD) meta-analysis evaluating the effectiveness of first-line ovulation induction for polycystic ovary syndrome (PCOS), IPD were only available from 20 studies of 53 randomized controlled trials (RCTs). We noticed that the summary effect sizes of meta-analyses of RCTs without IPD sharing were different from those of RCTs with IPD sharing. Granting access to IPD for secondary analysis has implications for promoting fair and transparent conduct of RCTs. It is, however, still common for authors to choose to withhold IPD, limiting the impact of and confidence in the results of RCTs and systematic reviews based on aggregate data. OBJECTIVE AND RATIONALE We performed a meta-epidemiologic study to elucidate if RCTs without IPD sharing have lower quality and more methodological issues than those with IPD sharing in an IPD meta-analysis evaluating first-line ovulation induction for PCOS. SEARCH METHODS We included RCTs identified for the IPD meta-analysis. We dichotomized RCTs according to whether they provided IPD (shared group) or not (non-shared group) in the IPD meta-analysis. We restricted RCTs to full-text published trials written in English. We assessed and compared RCTs in the shared and non-shared groups on the following criteria: Risk of Bias (RoB 2.0), GRADE approach, adequacy of trial registration; description of statistical methods and reproducibility of univariable statistical analysis; excessive similarity or difference in baseline characteristics that is not compatible with chance; and other miscellaneous methodological issues. OUTCOMES In total, 45 trials (8697 women) were included in this study. IPD were available from 17 RCTs and 28 trials were categorized as the non-shared IPD group. Pooled risk rates obtained from the shared and non-shared groups were different. Overall low risk of bias was associated with 13/17 (76%) of shared RCTs versus 7/28 (25%) of non-shared RCTs. For RCTs that started recruitment after 1 July 2005, adequate trial registration was found in 3/9 (33%) of shared IPD RCTs versus 0/16 (0%) in non-shared RCTs. In total, 7/17 (41%) of shared RCTs and 19/28 (68%) of non-shared RCTs had issues with the statistical methods described. The median (range) of inconsistency rate per study, between reported and reproduced analyses for baseline variables, was 0% (0–92%) (6 RCTs applicable) in the shared group and 54% (0–100%) (13 RCTs applicable) in the non-shared group. The median (range) of inconsistency rate of univariable statistical results for the outcome(s) per study was 0% (0–63%) (14 RCTs applicable) in the shared group and 44% (0–100%) (24 RCTs applicable) in the non-shared group. The distributions of simulation-generated P-values from comparisons of baseline continuous variables between intervention and control arms suggested that RCTs in the shared group are likely to be consistent with properly conducted randomization (P = 0.163), whereas this was not the case for the RCTs in the non-shared group (P = 4.535 × 10−8). WIDER IMPLICATIONS IPD meta-analysis on evaluating first-line ovulation induction for PCOS preserves validity and generates more accurate estimates of risk than meta-analyses using aggregate data, which enables more transparent assessments of benefits and risks. The availability of IPD and the willingness to share these data may be a good indicator of quality, methodological soundness and integrity of RCTs when they are being considered for inclusion in systematic reviews and meta-analyses.