START Foundation: Coping with Bias and Fairness when Implementing and Using an AI System

Abstract

START Foundation annually supports nearly 200 young people with a migratory background through an individualized scholarship program aimed at personal development and identity building. In 2022, START Foundation expanded its reach by launching a digital learning platform, which faced challenges such as information overload and navigation difficulties due to a growing number of courses and a lack of individual support. To improve user experience on the digital learning platform, START Foundation explored implementing a recommender system for personalized course navigation based on methods from the field of artificial intelligence (AI). However, this exploration raised ethical concerns, particularly regarding bias and fairness in AI systems. In light of these concerns, the following questions emerged: Should START Foundation integrate the AI-based recommender system into its digital learning platform? If so, what factors should START Foundation consider regarding bias and fairness in its AI-based recommender system in order to prevent negative consequences for its scholars? Readers are encouraged to explore the interactive learning module on bias and fairness in AI systems, available at https://bias-and-fairness-in-ai-systems.de/en/home/.