Novel Tools for Single Comparative and Unified Evaluation of Qualitative and Quantitative Bioassays: SS/PV–ROC and SS-J/PV-PSI Index–ROC Curves with Integrated Concentration Distributions, and SS-J/PV-PSI Index Cut-Off Diagrams

Abstract

Background: This investigation is both a study of potential non-invasive diagnostic approaches for the bladder cancer biomarker UBC® Rapid test and a study including novel comparative methods for bioassay evaluation and comparison that uses bladder cancer as a useful example. The objective of the paper is not to investigate specific data. It is used only for demonstration, partially to compare ROC methodologies and also to show how both sensitivity/specificity and predictive values can be used in clinical diagnostics and decision making. This study includes ROC curves with integrated cut-off distribution curves for a comparison of sensitivity/specificity (SS) and positive/negative predictive values (PPV/NPV or PV), as well as SS-J index/PV-PSI index–ROC curves and SS-J/PV-PSI index cut-off diagrams (J = Youden, PSI = Predictive Summary Index) for the unified direct comparison of SS-J/PV results achieved via quantitative and/or qualitative bioassays and an identification of optimal separate or unified index cut-off points. Patients and Methods: According to the routine diagnostics, there were 91 patients with confirmed bladder cancer and 1152 patients with no evidence of bladder cancer, leading to a prevalence value of 0.073. This study performed a quantitative investigation of used-up test cassettes from the visual UBC® Rapid qualitative point-of-care assay, which had already been applied in routine diagnostics. Using a photometric reader, quantitative data could also be obtained from the test line of the used cassettes. Interrelations between SS and PV values were evaluated using cumulative distribution analysis (CAD), SS/PV–ROC curves, SS-J/PV-PSI index–ROC curves, and the SS-J/PV-PSI index cut-off diagram. The maximum unified SS-J/PV-PSI index value and its corresponding cut-off value were determined and calculated with the SS-J/PV-PSI index cut-off diagram. Results: The use of SS/PV–ROC curves with integrated cut-off concentration distribution curves provides improved diagnostic information compared to “traditional” ROC curves. The threshold distributions integrated as curves into SS/PV–ROC curves and SS-J/PV-PSI index–ROC curves run in opposite directions. In contrast to the SS–ROC curves, the PV–ROC and the novel PV-PSI index–ROC curves had neither an area under the curve (AUC) nor a range from 0% to 100%. The cut-off level of the qualitative assay was 7.5 µg/L, with a sensitivity of 65.9% and a specificity of 63.3%, and the PPV was 12.4% and the NPV was 95.9%, at a threshold value of 12.5 µg/L. Based on these set concentrations, the reader-based evaluation revealed a graphically estimated 5% increase in sensitivity and a 13% increase in specificity, as compared to the visual qualitative POC test. In the case of predictive values, there was a gain of 8% for PPV and 10% for NPV. The index values and cut-offs were as follows: visual SS-J index, 0.328 and 35 µg/L; visual PV-PSI index, 0.083 and 5.4 µg/L; maximal reader Youden index, 0.0558 and 250 µg/L; and maximal PV-PSI index, 0.459 and 250 µg/L, respectively. The maximum unified SS-J/PV-PSI index value was 0.32, and the cut-off was 43 µg/L. The reciprocal SS-J index correctly detected one out of three patients, while the reciprocal PV-PSI index gave one out of twelve patients a correct diagnosis. Conclusions: ROC curves including cut-off distribution curves supplement the information lost in “traditionally plotted” ROC curves. The novel sets of ROC and index–ROC curves and the new SS/PV index cut-off diagrams enable the simultaneous comparison of sensitivity/specificity and predictive value profiles of diagnostic tools and the identification of optimal cut-off values at maximal index values, even in a unifying SS/PV approach. Because the curves within an SS-J/PV-PSI index cut-off diagram are distributed over the complete cut-off range of a quantitative assay, this field is open for special clinical considerations, with the need to vary the mentioned clinical diagnostic parameters. Complete or partial areas over the x-axis (AOX) can be calculated for summarized quantitative or qualitative effectivity evaluations with respect to single and/or unified SS-J and PV-PSI indices and with respect to single, several, or several unified assays. The SS-J/PV-PSI index-AOX approach is a new tool providing additional joint clinical information, and the reciprocal SS-J indices can predict the number of patients with a correct diagnosis and the number of persons who need to be examined in order to correctly predict a diagnosis of the disease. These methods could be used in applications like medical or plant epidemiology, machine learning algorithms, and neural networks.