Deep Learning Models for Analyzing Social Construction of Knowledge Online

Abstract

Gunawardena et al.’s (1997) Interaction Analysis Model (IAM) is one of the most frequently employed frameworks to guide the qualitative analysis of social construction of knowledge online. However, qualitative analysis is time consuming, and precludes immediate feedback to revise online courses while being delivered. To expedite analysis with a large dataset, this study explores how two neural network architectures—a feed-forward network (Doc2Vec) and a large language model transformer (BERT)—could automatically predict phases of knowledge construction using IAM. The methods interrogated the extent to which the artificial neural networks’ predictions of IAM Phases approximated a human coder’s qualitative analysis. Key results indicate an accuracy of 21.55% for Doc2Vec phases I-V, 43% for fine-tuning a pre-trained large language model (LLM), and 52.79% for prompt-engineering an LLM. Future studies for improving accuracy should consider either training the models with larger datasets or focusing on the design of prompts to improve classification accuracy. Grounded on social constructivism and IAM, this study has implications for designing and supporting online collaborative learning where the goal is social construction of knowledge. Moreover, it has teaching implications for guiding the design of AI tools that provide beneficial feedback for both students and course designers.