Artificial intelligence for health message generation: an empirical study using a large language model (LLM) and prompt engineering

Abstract

IntroductionThis study introduces and examines the potential of an AI system to generate health awareness messages. The topic of folic acid, a vitamin that is critical during pregnancy, served as a test case.MethodWe used prompt engineering to generate awareness messages about folic acid and compared them to the most retweeted human-generated messages via human evaluation with an university sample and another sample comprising of young adult women. We also conducted computational text analysis to examine the similarities between the AI-generated messages and human generated tweets in terms of content and semantic structure.ResultsThe results showed that AI-generated messages ranked higher in message quality and clarity across both samples. The computational analyses revealed that the AI generated messages were on par with human-generated ones in terms of sentiment, reading ease, and semantic content.DiscussionOverall, these results demonstrate the potential of large language models for message generation. Theoretical, practical, and ethical implications are discussed.