Assessing the Use of GitHub Copilot on Students of Engineering of Information Systems

Abstract

This study examines the impact of AI programming assistants like GitHub Copilot and ChatGPT on software engineering efficiency, an area that has seen limited empirical research. We experimentally evaluated the performance of programmers ([Formula: see text]) in Python coding tasks with and without AI assistance, measuring time-to-completion and feature implementation. Results indicate that participants utilizing AI assistance completed tasks significantly faster (p = 0.033) and implemented more required features (p = 0.012) compared to those relying solely on unaided coding. These findings offer empirical insights into the integration of AI tools in software development workflows, highlighting their potential to enhance efficiency without compromising code quality or completeness, with implications for organizational pipelines and practitioner skills. Responses to exit surveys suggest that participants without IA tools assistance encountered frustrations related to code recall, time constraints, and problem-solving, while assisted participants reported no negative experiences, focusing instead on successful completion of tasks within the allotted time.