geck: trio-based comparative benchmarking of variant calls

Abstract

MotivationClassical methods of comparing the accuracies of variant calling pipelines are based on truth sets of variants whose genotypes are previously determined with high confidence. An alternative way of performing benchmarking is based on Mendelian constraints between related individuals. Statistical analysis of Mendelian violations can provide truth set-independent benchmarking information, and enable benchmarking less-studied variants and diverse populations.ResultsWe introduce a statistical mixture model forcomparing two variant calling pipelines from genotype data they produce after running on individual members of a trio. We determine the accuracy of our model by comparing the precision and recall of GATK Unified Genotyper and Haplotype Caller on the high-confidence SNPs of the NIST Ashkenazim trio and the two independent Platinum Genome trios. We show that our method is able to estimate differential precision and recall between the two pipelines with 10-3 uncertainty.AvailabilityThe Python library geck, and usage examples are available at the following URL: https://github.com/sbg/geckContactpeter.komar@sbgdinc.comSupplementary informationSupplementary materials are available at bioRxiv.