Affiliation:
1. Department of Computers, College of Engineering and Computing Al Qunfidhah, Umm Al-Qura University, Makkah 24382, Saudi Arabia
2. Department of Computer Science and Engineering, Egypt-Japan University of Science and Technology, Alexandria 21934, Egypt
Abstract
There is a pressing need for healthcare conversational agents with domain-specific expertise to ensure the provision of accurate and reliable information tailored to specific medical contexts. Moreover, there is a notable gap in research ensuring the credibility and trustworthiness of the information provided by these healthcare agents, particularly in critical scenarios such as medical emergencies. Pilgrims come from diverse cultural and linguistic backgrounds, often facing difficulties in accessing medical advice and information. Establishing an AI-powered multilingual chatbot can bridge this gap by providing readily available medical guidance and support, contributing to the well-being and safety of pilgrims. In this paper, we present a comprehensive methodology aimed at enhancing the reliability and efficacy of healthcare conversational agents, with a specific focus on addressing the needs of Hajj pilgrims. Our approach leverages domain-specific fine-tuning techniques on a large language model, alongside synthetic data augmentation strategies, to optimize performance in delivering contextually relevant healthcare information by introducing the HajjHealthQA dataset. Additionally, we employ a retrieval-augmented generation (RAG) module as a crucial component to validate uncertain generated responses, which improves model performance by 5%. Moreover, we train a secondary AI agent on a well-known health fact-checking dataset and use it to validate medical information in the generated responses. Our approach significantly elevates the chatbot’s accuracy, demonstrating its adaptability to a wide range of pilgrim queries. We evaluate the chatbot’s performance using quantitative and qualitative metrics, highlighting its proficiency in generating accurate responses and achieve competitive results compared to state-of-the-art models, in addition to mitigating the risk of misinformation and providing users with trustworthy health information.
Reference40 articles.
1. Abdelmoety, D., El-Bakri, N., Almowalld, W., Turkistani, Z., Bugis, B., Baseif, E., Melbari, M.H., AlHarbi, K., and Abu-Shaheen, A. (2018). Characteristics of Heat Illness during Hajj: A Cross-Sectional Study. BioMed Res. Int., 2018.
2. Determining the types of diseases and emergency issues in Pilgrims during Hajj: A literature review;Bakar;Int. J. Adv. Comput. Sci. Appl.,2016
3. Infectious diseases and preventive measures during hajj mass gatherings: A review of the literature;Razavi;Arch. Clin. Infect. Dis.,2018
4. Hajj-associated infections;Piednoir;MEdecine Mal. Infect.,2016
5. Yezli, S., Yassin, Y., Mushi, A., Almuzaini, Y., and Khan, A. (2022). Pattern of utilization, disease presentation, and medication prescribing and dispensing at 51 primary healthcare centers during the Hajj mass gathering. BMC Health Serv. Res., 22.