Affiliation:
1. National University of Singapore, Republic of Singapore
2. Ulsan National Institute of Science and Technology, South Korea
Abstract
Automated feedback generation for introductory programming assignments is useful for programming education. Most works try to generate feedback to correct a student program by comparing its behavior with an instructor’s reference program on selected tests. In this work, our aim is to generate verifiably correct program repairs as student feedback. A student-submitted program is aligned and composed with a reference solution in terms of control flow, and the variables of the two programs are automatically aligned via predicates describing the relationship between the variables. When verification attempt for the obtained aligned program fails, we turn a verification problem into a MaxSMT problem whose solution leads to a minimal repair. We have conducted experiments on student assignments curated from a widely deployed intelligent tutoring system. Our results show that generating verified repair without sacrificing the overall repair rate is possible. In fact, our implementation, Verifix, is shown to outperform Clara, a state-of-the-art tool, in terms of repair rate. This shows the promise of using verified repair to generate high confidence feedback in programming pedagogy settings.
Funder
Singapore Ministry of Education (MoE) Tier 3
National Research Foundation of Korea
Korea government
Institute for Information & Communications Technology Planning & Evaluation
Publisher
Association for Computing Machinery (ACM)
Cited by
9 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Flexible control flow graph alignment for delivering data-driven feedback to novice programming learners;Journal of Systems and Software;2024-04
2. A Survey of Learning-based Automated Program Repair;ACM Transactions on Software Engineering and Methodology;2023-12-23
3. How Helpful do Novice Programmers Find the Feedback of an Automated Repair Tool?;2023 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE);2023-11-28
4. Using Large Language Models for Bug Localization and Fixing;2023 12th International Conference on Awareness Science and Technology (iCAST);2023-11-09
5. Toward AI-assisted Exercise Creation for First Course in Programming through Adversarial Examples of AI Models;2023 IEEE 35th International Conference on Software Engineering Education and Training (CSEE&T);2023-08