A Survey of Automated Programming Hint Generation: The HINTS Framework

Abstract

Automated tutoring systems offer the flexibility and scalability necessary to facilitate the provision of high-quality and universally accessible programming education. To realise the potential of these systems, recent work has proposed a diverse range of techniques for automatically generating feedback in the form of hints to assist students with programming exercises. This article integrates these apparently disparate approaches into a coherent whole. Specifically, it emphasises that all hint techniques can be understood as a series of simpler components with similar properties. Using this insight, it presents a simple framework for describing such techniques, the Hint Iteration by Narrow-down and Transformation Steps framework, and surveys recent work in the context of this framework. Findings from this survey include that (1) hint techniques share similar properties, which can be used to visualise them together, (2) the individual steps of hint techniques should be considered when designing and evaluating hint systems, (3) more work is required to develop and improve evaluation methods, and (4) interesting relationships, such as the link between automated hints and data-driven evaluation, should be further investigated. Ultimately, this article aims to facilitate the development, extension, and comparison of automated programming hint techniques to maximise their educational potential.