Affiliation:
1. School of Fashion and Textiles, Hong Kong Polytechnic University, Hong Kong, China
2. School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China
3. Engineering Research Center of Integration and Application of Digital Learning Technology, Ministry of Education, Beijing 100039, China
Abstract
Optimizing spaced repetition schedules is of great importance for enhancing long-term memory retention in both real-world applications, e.g., online learning platforms, and academic applications, e.g., cognitive science. Traditional methods tackle this problem by employing handcrafted rules while modern methods try to optimize scheduling using deep reinforcement learning (DRL). Existing DRL-based approaches model the problem by selecting the optimal next item to appear, which implies the learner can only learn one item in a day. However, the most essential point to enhancing long-term memory is to select the optimal interval to review. To this end, we present a novel approach to DRL to optimize spaced repetition scheduling. The contribution of our framework is three-fold. We first introduce a Transformer-based model to estimate the recall probability of a learning item accurately, which encodes the temporal dynamics of a learner’s learning trajectories. Second, we build a simulation environment based on our recall probability estimation model. Third, we utilize the Deep Q-Network (DQN) as the agent to learn the optimal review intervals for learning items and train the policy in a recurrent manner. Experimental results demonstrate that our framework achieves state-of-the-art performance against competing methods. Our method achieves an MAE (mean average error) score of 0.0274 on a memory prediction task, which is 11% lower than the second-best method. For spaced repetition scheduling, our method achieves mean recall probabilities of 0.92, 0.942, and 0.372 in three different environments, the best performance in all scenarios.
Funder
Innovation Fund of the Engineering Research Center of the Ministry of Education for the Integration and Application of Digital Learning Technology
Reference35 articles.
1. Spacing effects in learning: A temporal ridgeline of optimal retention;Cepeda;Psychol. Sci.,2008
2. Finke, L. (2023). Markov Models for Spaced Repetition Learning, Mathematisches Institut der Georg-August-Universität Göttingen.
3. Memory: A contribution to experimental psychology;Ebbinghaus;Ann. Neurosci.,2013
4. A memory schedule;Pimsleur;Mod. Lang. J.,1967
5. Leitner, S., and Totter, R. (1972). So Lernt Man Lernen, Herder. Angewandte Lernpsychologie ein Weg zum Erfolg.