eCV: Enhanced coefficient of variation and IDR extensions for reproducibility assessment of high-throughput experiments with multiple replicates

Abstract

ABSTRACTMotivationReproducibility assessment is essential in extracting reliable scientific insights from highthroughput experiments. Inconsistency between technical replicates poses a challenge, particularly clear in next generation sequencing technologies based on immunoprecipitations, where the assessment of reproducibility in peak identification is a critical analytical step. While the Irreproducibility Discovery Rate (IDR) method has been instrumental in assessing reproducibility, its standard implementation is constrained to handling only two replicates. In the current era of steadily growing sample sizes, eased by multiplexing and reduced sequencing costs, highly performing methods that handle any number of replicates are desirable.ResultsWe introduce three novel methods for reproducibility assessment in high-throughput data that handle an arbitrary number of replicates. The first, general IDR (gIDR), extends the standard IDR by adapting its Expectation-Maximization (EM) algorithm to handle distributions of any dimensions dictated by the number of replicates. The second, meta-IDR (mIDR), employs a meta-analysis approach, calculating local IDR scores for all pairs of replicates and combining them using standard probability rules. The third method introduces an “enhanced” Coefficient of Variation (eCV), ranking features based on intensity and variability, using a parametric bootstrap approach to obtain an index analogous to local IDR. Comparative analysis with traditional IDR in simulated and experimental data reveals the heightened performance of the proposed multivariate alternatives under varying scenarios, thereby addressing the critical challenge of reproducibility assessment in contemporary high-throughput experiments.Availability and implementationThe described methods are implemented as an R package:https://github.com/eclipsebio/eCVContactinfo@eclipsebio.com