Implementation and evaluation of an adaptive online summer preparatory course for general chemistry: Whom does it benefit?

Abstract

General chemistry courses are key gateways for many Science, Technology, Engineering, and Mathematics (STEM) majors. Here, we report on the implementation and evaluation of an adaptive, ALEKS-based online preparatory module (PM) for general chemistry. The module was made available in Summer 2018 at no cost to all students entering any section of general chemistry that fall. Of the 827 students who registered into the PM, 44% fully completed the module, 48% completed part of the module, and 8% did not complete any of the module. Considering students enrolled in first-term general chemistry, we find a marked increase in ACS final exam percentile for students who completed more than 50% of the module. This is suggested to reflect the self-selection of users who were highly motivated and/or likely to succeed in the course, a hypothesis supported by an analysis using an internal diagnostic metric, the predicted first-year quality point average (PQPA). To examine longer term impacts of the PM, we examined performance in subsequent chemistry courses, through second-semester organic, and found that students completing more than 50% of the module outperformed their counterparts across all courses, with the gap largest in first semester general chemistry and narrowing across subsequent courses. Finally, we surveyed students in summer 2020, two years after the PM offering. The survey indicated overall satisfaction with the PM. For students who did not complete the module, primary reasons given were difficulty (29% of respondents) and insufficient time (46%). As the module did not proportionally benefit the target group of underprepared or at-risk students, we suggest tweaks for future implementations.