They shall be fair, transparent, and robust: auditing learning analytics systems

Abstract

AbstractIn the near future, systems, that use Artificial Intelligence (AI) methods, such as machine learning, are required to be certified or audited for fairness if used in ethically sensitive fields such as education. One example of those upcoming regulatory initiatives is the European Artificial Intelligence Act. Interconnected with fairness are the notions of system transparency (i.e. how understandable is the system) and system robustness (i.e. will similar inputs lead to similar results). Ensuring fairness, transparency, and robustness requires looking at data, models, system processes, and the use of systems as the ethical implications arise at the intersection between those. The potential societal consequences are domain specific, it is, therefore, necessary to discuss specifically for Learning Analytics (LA) what fairness, transparency, and robustness mean and how they can be certified. Approaches to certifying and auditing fairness in LA include assessing datasets, machine learning models, and the end-to-end LA process for fairness, transparency, and robustness. Based on Slade and Prinsloo’s six principals for ethical LA, relevant audit approaches will be deduced. Auditing AI applications in LA is a complex process that requires technical capabilities and needs to consider the perspectives of all stakeholders. This paper proposes a comprehensive framework for auditing AI applications in LA systems from the perspective of learners' autonomy, provides insights into different auditing methodologies, and emphasizes the importance of reflection and dialogue among providers, buyers, and users of these systems to ensure their ethical and responsible use.