Abstract
AbstractEducational data mining (EDM) can be used to design better and smarter learning technology by finding and predicting aspects of learners. Amend if necessary. Insights from EDM are based on data collected from educational environments. Among these educational environments are computer-based educational systems (CBES) such as learning management systems (LMS) and conversational intelligent tutoring systems (CITSs). The use of large language models (LLMs) to power a CITS holds promise due to their advanced natural language understanding capabilities. These systems offer opportunities for enriching management and entrepreneurship education. Collecting data from classes experimenting with these new technologies raises some ethical challenges. This paper presents an EDM framework for analyzing and evaluating the impact of these LLM-based CITS on learning experiences in management and entrepreneurship courses and also places strong emphasis on ethical considerations. The different learning experience aspects to be tracked are (1) learning outcomes and (2) emotions or affect and sentiments. Data sources comprise Learning Management System (LMS) logs, pre-post-tests, and reflection papers gathered at multiple time points. This framework aims to deliver actionable insights for course and curriculum design and development through design science research (DSR), shedding light on the LLM-based system’s influence on student learning, engagement, and overall course efficacy. Classes targeted to apply this framework have 30–40 students on average, grouped between 2 and 6 members. They will involve sophomore to senior students aged 18–22 years. One entire semester takes about 14 weeks. Designed for broad application across diverse courses in management and entrepreneurship, the framework aims to ensure that the utilization of LLMs in education is not only effective but also ethically sound.
Publisher
Springer Nature Singapore
Reference18 articles.
1. Baker RS (2014) Educational data mining: an advance for intelligent systems in education. IEEE Intell Syst 29(3):78–82. IEEE
2. Motz B, Quick J, Schroeder N, Zook J, Gunkel M (2019) The validity and utility of activity logs as a measure of student engagement. In: Proceedings of the 9th international conference on learning analytics and knowledge, pp 300–309. ACM, Tempe AZ USA. https://dl.acm.org/doi/10.1145/3303772.3303789, 11 citations (Crossref) [2023-09-30]
3. Romero C, Ventura S (2013) Data mining in education. Wiley Interdisc Rev Data Mining Knowl Discov 3(1):12–27. Wiley
4. Holmes M, Latham A, Crockett K, O’Shea JD (2017) Tomorrow’s learning: involving everyone. learning with and about technologies and computing. In: 11th IFIP TC 3 world conference on computers in education, WCCE 2017, Dublin, Ireland, July 3–6, 2017, Revised Selected Papers. IFIP Advances in Information and Communication Technology, vol 515, pp 251–260. https://www.scopus.com/inward/record.uri?eid=2-s2.0-85041551804 &doi=10.1007%2f978-3-319-74310-3_27 &partnerID=40 &md5=d0f6929e9b416a1e6036891b725905ab, 2 citations (Crossref) [2023-07-11]
5. Baidoo-Anu D, Owusu Ansah L (2023) Education in the era of generative artificial intelligence (AI): understanding the potential benefits of ChatGPT in promoting teaching and learning. Available at SSRN 4337484