Developing Machine Learning Models for Automated Analysis of Organic Chemistry Students’ Written Descriptions of Organic Reaction Mechanisms

Abstract

Many assessments in organic chemistry ask students to produce reaction mechanisms with the electron-pushing formalism. It is well known that students can apply the electron-pushing formalism without engaging in chemical reasoning about the processes underlying mechanisms. Furthermore, engagement in mechanistic and causal reasoning correlates with student performance on organic chemistry tasks. Hence, it is valuable to elicit students' explanations of mechanisms beyond relying on traditional mechanism assessments. One evidence-based approach for encouraging and eliciting students' mechanistic explanations is through writing. However, instructors may hesitate to implement writing in their courses due to a lack of tools available to provide formative feedback on students' mechanistic explanations. To address this challenge, we analyzed students' written explanations of three different organic reaction mechanisms for individual features involved in mechanistic reasoning. In this chapter, we present our adaptation of Russ et al.'s mechanistic reasoning framework specifically for students' written explanations of organic chemistry reaction mechanisms. Additionally, we describe a set of predictive models which we have used to accurately identify features of students' writing involved in mechanistic reasoning in the context of the three different reaction mechanisms. This work has implications for instructors seeking to identify students' reasoning in written explanations of organic reaction mechanisms. Additionally, this work has implications for future research into developing immediate and automated student- and instructor-facing formative feedback to encourage students' development of mechanistic and causal reasoning.