Examining Discrimination Performance and Likelihood Ratio Values for Two Different Likelihood Ratio Systems Using the Provedit Dataset

Abstract

ABSTRACTThe conventional capillary electrophoresis (CE) genotyping workflow used in forensic DNA laboratories is composed of two processes: measurement and interpretation. The outcome of the measurement process is an electropherogram (EPG). The outcome of the interpretation process is a strength of evidence statement often reported in the form of a likelihood ratio (LR) which typically requires probabilistic genotyping software (PGS). An LR system is defined as the entire pipeline of the measurement and interpretation processes where PGS is a piece of the whole LR system. To gain understanding on how two LR systems perform, a total of 154 two-person mixture, 147 three-person mixture, and 127 four-person mixture profiles of varying DNA quality, DNA quantity, and mixture ratios were obtained from the filtered (.CSV) files of the GlobalFiler 29 cycles 15s PROVEDIt dataset and deconvolved in two independently developed fully continuous programs, STRmix v2.6 and EuroForMix v2.1.0. Various parameters were set in each software and LR computations obtained from the two software were based on same/fixed EPG features, same pair of propositions, number of contributors, theta, and population allele frequencies. The ability of each LR system to discriminate between contributor (H1-true) and non-contributor (H2-true) scenarios was evaluated qualitatively and quantitatively. Differences in the numeric LR values and their corresponding verbal classifications between the two LR systems were compared. The magnitude of the differences in the assigned LRs and the potential explanations for the observed differences greater than or equal to 3 on the log10 scale were described. Cases of LR < 1 for H1-true tests and LR > 1 for H2-true tests were also discussed. Our intent is to demonstrate the value of using a publicly available ground truth known mixture dataset to assess discrimination performance of any LR system and show the steps used to investigate and understand similarities and differences between different LR systems. We share our observations with the forensic community and describe how examining more than one PGS with similar discrimination power can be beneficial, help analysts compare interpretation especially with low-template profiles or minor contributor cases, and be a potential additional diagnostic check even if software in use does contain certain diagnostic statistics as part of the output.HighlightsThe use of two different Likelihood Ratio (LR) systems to assign LRs is discussed.H1-true and H2-true tests are performed using STRmix and EuroForMix and a large set of PROVEDIt mixture profiles.Assessment of discrimination performance of two LR systems using ROC plots, scatter plots, and relative frequency histograms.The ability of the two LR systems to discriminate between contributors and non-contributors are statistically indistinguishable for the data that we considered.Potential reasons for the differences in LR values between the two LR systems that are ≥ 3 on the log10 scale are investigated and discussed.Contributors with LRs < 1 and non-contributors with LRs > 1 generated from each LR system are discussed.