Disparities in seizure outcomes revealed by large language models

Author:

Xie Kevin12ORCID,Ojemann William K S12ORCID,Gallagher Ryan S23,Shinohara Russell T4,Lucas Alfredo123,Hill Chloé E5,Hamilton Roy H3,Johnson Kevin B1467,Roth Dan6,Litt Brian123,Ellis Colin A23ORCID

Affiliation:

1. Department of Bioengineering, University of Pennsylvania , Philadelphia, PA 19104, United States

2. Center for Neuroengineering and Therapeutics, University of Pennsylvania , Philadelphia, PA 19104, United States

3. Department of Neurology, University of Pennsylvania , Philadelphia, PA 19104, United States

4. Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania , Philadelphia, PA 19104, United States

5. Department of Neurology, University of Michigan , Ann Arbor, MI 48109, United States

6. Department of Computer and Information Science, University of Pennsylvania , Philadelphia, PA 19104, United States

7. Department of Pediatrics, University of Pennsylvania , Philadelphia, PA 19104, United States

Abstract

Abstract Objective Large-language models (LLMs) can potentially revolutionize health care delivery and research, but risk propagating existing biases or introducing new ones. In epilepsy, social determinants of health are associated with disparities in care access, but their impact on seizure outcomes among those with access remains unclear. Here we (1) evaluated our validated, epilepsy-specific LLM for intrinsic bias, and (2) used LLM-extracted seizure outcomes to determine if different demographic groups have different seizure outcomes. Materials and Methods We tested our LLM for differences and equivalences in prediction accuracy and confidence across demographic groups defined by race, ethnicity, sex, income, and health insurance, using manually annotated notes. Next, we used LLM-classified seizure freedom at each office visit to test for demographic outcome disparities, using univariable and multivariable analyses. Results We analyzed 84 675 clinic visits from 25 612 unique patients seen at our epilepsy center. We found little evidence of bias in the prediction accuracy or confidence of outcome classifications across demographic groups. Multivariable analysis indicated worse seizure outcomes for female patients (OR 1.33, P ≤ .001), those with public insurance (OR 1.53, P ≤ .001), and those from lower-income zip codes (OR ≥1.22, P  ≤ .007). Black patients had worse outcomes than White patients in univariable but not multivariable analysis (OR 1.03, P = .66). Conclusion We found little evidence that our LLM was intrinsically biased against any demographic group. Seizure freedom extracted by LLM revealed disparities in seizure outcomes across several demographic groups. These findings quantify the critical need to reduce disparities in the care of people with epilepsy.

Funder

National Institute of Neurological Disorders and Stroke

National Institutes of Health

Mirowski Family Foundation

National Science Foundation Research

American Academy of Neurology Susan S. Spencer Clinical Research Training Scholarship

Office of Naval Research

Publisher

Oxford University Press (OUP)

Cited by 2 articles. 订阅此论文施引文献 订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3