The Role of AI in Programming Education

Abstract

AI has garnered much attention from education researchers since the launch of ChatGPT-3.5 at the end of November 2022, and the subsequent development of ChatGPT-4 and other Large Language Models (LLMs) such as Copilot (Bing), Gemini (Bard) and Claude. Owing to its effectiveness as both a learning and teaching tool, AI could be considered one of the most important technologies for developing pedagogy across a wide range of disciplines. The emergence of prompt engineering has made it possible for both programming specialists and non-specialist users to benefit from using the technology. This study adopts a non-specialist (non-developer) perspective and explores LLM responses to both conversational and structured prompting with a focus on the teaching of recursive functions, specifically the Fibonacci series. Differences in the depth of explanation provided across the models indicate the need for precise integration of AI. A specially designed assessment rubric facilitated comparison of responses across the LLMs and revealed that ChatGPT-4 outperformed all other models in both prompt types. Although the effectiveness of structured prompting varied across models, when used in ChatGPT-4, this method resulted in a more relevant and in-depth assessment of student understanding than that obtained through conversational prompting. The study concludes with several recommendations for the use of multiple AI tools to improve the teaching of programming concepts and enhance students' educational experiences.