Art or Artifact: Evaluating the Accuracy, Appeal, and Educational Value of AI-Generated Imagery in DALL·E 3 for Illustrating Congenital Heart Diseases

Abstract

Abstract Artificial Intelligence (AI), particularly AI-Generated Imagery, holds the capability to transform medical and patient education. This research explores the use of AI-generated imagery, from text-to-images, in medical education, focusing on congenital heart diseases (CHD). Utilizing ChatGPT's DALL·E 3, the research aims to assess the accuracy and educational value of AI-created images for 20 common CHDs. The study involved generating a total of 110 images for normal human heart and 20 common CHDs through DALL·E 3. Then, 33 healthcare professionals systematically assessed these AI-generated images by variable levels of healthcare professionals (HCPs) using a developed framework to individually assess each image anatomical accuracy, in-picture text usefulness, image appeal to medical professionals and the potential to use the image in medical presentations. Each item was assessed on a Likert scale of three. The assessments produced a total of 3630 images’ assessments. Most AI-generated cardiac images were rated poorly as follows: 80.8% of images were rated as anatomically incorrect or fabricated, 85.2% rated to have incorrect text labels, 78.1% rated as not usable for medical education. The nurses and medical interns were found to have a more positive perception about the AI-generated cardiac images compared to the faculty members, pediatricians, and cardiology experts. Complex congenital anomalies were found to be significantly more predicted to anatomical fabrication compared to simple cardiac anomalies. There were significant challenges identified in image generation. These findings suggest adopting a cautious approach in integrating AI imagery in medical education, emphasizing the need for rigorous validation and interdisciplinary collaboration. The study advocates for future AI-models to be fine-tuned with accurate medical data, enhancing their reliability and educational utility.