Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information-Reference-Cited by-同舟云学术

Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information

Published:2023-03-01 Issue:2 Volume:7 Page:
ISSN:2515-5091
Container-title:JNCI Cancer Spectrum
language:en
Short-container-title:

Author:

Johnson Skyler B¹²^ORCID,King Andy J²³,Warner Echo L²⁴,Aneja Sanjay⁵⁶,Kann Benjamin H⁷,Bylund Carma L⁸

Affiliation:

1. Department of Radiation Oncology, University of Utah School of Medicine, Huntsman Cancer Institute , Salt Lake City, UT, USA

2. Cancer Control and Population Sciences, Huntsman Cancer Institute , Salt Lake City, UT, USA

3. Department of Communication, University of Utah , Salt Lake City, UT, USA

4. College of Nursing, University of Utah , Salt Lake City, UT, USA

5. Center for Outcomes Research and Evaluation, Yale School of Medicine , New Haven, CT, USA

6. Department of Therapeutic Radiology, Yale School of Medicine , New Haven, CT, USA

7. Department of Radiation Oncology, Dana-Farber Cancer Institute/Brigham and Women’s Hospital, Harvard Medical School , Boston, MA, USA

8. Department of Health Outcomes and Biomedical Informatics, University of Florida College of Medicine , Gainesville, FL, USA

Abstract

AbstractData about the quality of cancer information that chatbots and other artificial intelligence systems provide are limited. Here, we evaluate the accuracy of cancer information on ChatGPT compared with the National Cancer Institute’s (NCI’s) answers by using the questions on the “Common Cancer Myths and Misconceptions” web page. The NCI’s answers and ChatGPT answers to each question were blinded, and then evaluated for accuracy (accurate: yes vs no). Ratings were evaluated independently for each question, and then compared between the blinded NCI and ChatGPT answers. Additionally, word count and Flesch-Kincaid readability grade level for each individual response were evaluated. Following expert review, the percentage of overall agreement for accuracy was 100% for NCI answers and 96.9% for ChatGPT outputs for questions 1 through 13 (ĸ = ‒0.03, standard error = 0.08). There were few noticeable differences in the number of words or the readability of the answers from NCI or ChatGPT. Overall, the results suggest that ChatGPT provides accurate information about common cancer myths and misconceptions.

Funder

Huntsman Cancer Institute

Publisher

Oxford University Press (OUP)

Subject

Cancer Research,Oncology

Link

https://academic.oup.com/jncics/article-pdf/7/2/pkad015/49535391/pkad015.pdf

Reference10 articles.

1. Social and demographic patterns of health-related Internet use among adults in the United States: a secondary data analysis of the Health Information National Trends Survey;Calixte;Int J Environ Res Public Health,2020

2. Cancer misinformation and harmful information on Facebook and other social media: a brief report;Johnson;J Natl Cancer Inst,2022

3. Chatbots in the fight against the COVID-19 pandemic;Miner;NPJ Digit Med,2020