Affiliation:
1. Brown University, Providence, USA
Abstract
This paper documents a year-long experiment to “profile” the process of learning a programming language: gathering data to understand what makes a language hard to learn, and using that data to improve the learning process. We added interactive quizzes to The Rust Programming Language, the official textbook for learning Rust. Over 13 months, 62,526 readers answered questions 1,140,202 times. First, we analyze the trajectories of readers. We find that many readers drop-out of the book early when faced with difficult language concepts like Rust’s ownership types. Second, we use classical test theory and item response theory to analyze the characteristics of quiz questions. We find that better questions are more conceptual in nature, such as asking why a program does not compile vs. whether a program compiles. Third, we performed 12 interventions into the book to help readers with difficult questions. We find that on average, interventions improved quiz scores on the targeted questions by +20
Funder
National Science Foundation
Publisher
Association for Computing Machinery (ACM)
Reference36 articles.
1. Does Answering Higher-Level Questions While Reading Facilitate Productive Learning?
2. Jeremy Avigad Leonardo de Moura Soonho Kong and Sebastian Ullrich. 2023. Theorem Proving in Lean 4. https://lean-lang.org/theorem_proving_in_lean4/
3. Frank B. Baker. 2001. The Basics of Item Response Theory (2 ed.). ERIC Clearinghouse on Assessment and Evaluation, College Park, MD.
4. Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
5. Pyro: Deep Universal Probabilistic Programming;Bingham Eli;Journal of Machine Learning Research,2019