Author:
MacDonald Samual,Foley Helena,Yap Melvyn,Johnston Rebecca L.,Steven Kaiah,Koufariotis Lambros T.,Sharma Sowmya,Wood Scott,Addala Venkateswar,Pearson John V.,Roosta Fred,Waddell Nicola,Kondrashova Olga,Trzaskowski Maciej
Abstract
AbstractUncertainty estimation is crucial for understanding the reliability of deep learning (DL) predictions, and critical for deploying DL in the clinic. Differences between training and production datasets can lead to incorrect predictions with underestimated uncertainty. To investigate this pitfall, we benchmarked one pointwise and three approximate Bayesian DL models for predicting cancer of unknown primary, using three RNA-seq datasets with 10,968 samples across 57 cancer types. Our results highlight that simple and scalable Bayesian DL significantly improves the generalisation of uncertainty estimation. Moreover, we designed a prototypical metric—the area between development and production curve (ADP), which evaluates the accuracy loss when deploying models from development to production. Using ADP, we demonstrate that Bayesian DL improves accuracy under data distributional shifts when utilising ‘uncertainty thresholding’. In summary, Bayesian DL is a promising approach for generalising uncertainty, improving performance, transparency, and safety of DL models for deployment in the real world.
Funder
Cooperative Research Centres, Australian Government Department of Industry
Australian Research Council Industrial Transformation Training Centre for Information Resilience
National Health and Medical Research Council
NHMRC Emerging Leader 1 Investigator Grant
Publisher
Springer Science and Business Media LLC
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献