Abstract
ABSTRACTObjectiveTo propose a novel approach for enhancing clinical prediction models by combining structured and unstructured data with multimodal data fusion.MethodsWe presented a comprehensive framework that integrated multimodal data sources, including textual clinical notes, structured electronic health records (EHRs), and relevant clinical data from National Electronic Injury Surveillance System (NEISS) datasets. We proposed a novel hybrid fusion method, which incorporated state-of-the-art pre-trained language model, to integrate unstructured clinical text with structured EHR data and other multimodal sources, thereby capturing a more comprehensive representation of patient information.ResultsThe experimental results demonstrated that the hybrid fusion approach significantly improved the performance of clinical prediction models compared to traditional fusion frameworks and unimodal models that rely solely on structured data or text information alone. The proposed hybrid fusion system with RoBERTa language encoder achieved the best prediction of the Top 1 injury with an accuracy of 75.00% and Top 3 injuries with an accuracy of 93.54%.ConclusionOur study highlights the potential of integrating natural language processing (NLP) techniques with multimodal data fusion for enhancing clinical prediction models’ performances. By leveraging the rich information present in clinical text and combining it with structured EHR data, the proposed approach can improve the accuracy and robustness of predictive models. The approach has the potential to advance clinical decision support systems, enable personalized medicine, and facilitate evidence-based health care practices. Future research can further explore the application of this hybrid fusion approach in real-world clinical settings and investigate its impact on improving patient outcomes.
Publisher
Cold Spring Harbor Laboratory
Reference26 articles.
1. Predicting mortality in critically ill patients with diabetes using machine learning and clinical notes;BMC Medical Informatics and Decision Making,2020
2. Aramaki, E. , et al., Extraction of adverse drug effects from clinical records, in MEDINFO 2010. 2010, IOS Press. p. 739–743.
3. Devlin, J. , et al., Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
4. BioBERT: a pre-trained biomedical language representation model for biomedical text mining
5. Liao, K.P. , et al. , Development of phenotype algorithms using electronic medical records and incorporating natural language processing . bmj, 2015. 350.
Cited by
2 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献