A Generative AI Chatbot in High School Advising: A Qualitative Analysis of Domain-Specific Chatbot and ChatGPT

Abstract

Abstract Due to the variety of chatbot types and classifications, students and advisers may experience confusion when trying to select the right chatbot that can more trust it, however, the classification of chatbots depends on different factors including, the complexity of the task, the response-based approach and the type of the domain. Since selecting the most effective chatbot is crucial for high schools and students, a semi-structured interviews in qualitative research were conducted with eight high school students in order to investigate the students ‘perspectives on different seven responses of generative questions from the domain-specific chatbot named HSGAdviser, comparing it with the ChatGPT. All questions were related to students’ advising interests including university applications, admission tests, majors and more. The transcribed data were reviewed and examined by using the thematic analysis. However, the results reveal that most students found that HSGAdviser chatbot is easier, shorter, faster and more concise compared to ChatGPT, especially for Yes/No questions as students expect brief answers. However, some students found that certain crucial questions that can have a significance impact on their future, they would prefer the ChatGPT for more detailed information. The limitation of this study is the limited size of the participants. Nevertheless, in the future research, other high school students from different regions will participate in the study.