Abstract
AbstractObjectiveThis study evaluates an AI assistant developed using OpenAI’s GPT-4 for interpreting pharmacogenomic (PGx) testing results, aiming to improve decision-making and knowledge sharing in clinical genetics, and to enhance patient care with equitable access.MethodsThe AI assistant employs Retrieval Augmented Generation (RAG) combining retrieval and generative techniques. It employs a Knowledge Base (KB) comprising Clinical Pharmacogenetics Implementation Consortium (CPIC) data, with context-aware GPT-4 generating tailored responses to user queries from this KB, refined through prompt engineering and guardrails.ResultsEvaluated against a specialized PGx question catalog, the AI assistant showed high efficacy in addressing user queries. Compared with OpenAI’s ChatGPT 3.5, it demonstrated better performance, especially in provider-specific queries requiring specialized data and citations. Key areas for improvement include enhancing accuracy, relevancy, and representative language in responses.DiscussionThe integration of context-aware GPT-4 with RAG significantly enhanced the AI assistant’s utility. RAG’s ability to incorporate domain-specific CPIC data, including recent literature, proved beneficial. Challenges persist, such as the need for specialized genetic/PGx models to improve accuracy and relevancy and addressing ethical, regulatory, and safety concerns.ConclusionThis study underscores generative AI’s potential for transforming healthcare provider support and patient accessibility to complex pharmacogenomic information. While careful implementation of large language models like GPT-4 is necessary, it is clear that they can substantially improve understanding of pharmacogenomic data. With further development, these tools could augment healthcare expertise, provider productivity, and the delivery of equitable, patient-centered healthcare services.
Publisher
Cold Spring Harbor Laboratory