Already in 2022: Educational AI needs more than an appearance of objectivity

Ryan S. Baker and Aaron Hawn’s “Algorithmic Bias in Education” examines the causes and documented consequences of algorithmic bias in education. The article appeared in the December 2022 journal issue; its DOI points to an earlier online publication. It is thus an early reference from before the widespread use of generative AI. It primarily examines predictive and decision-making systems, rather than today’s chatbots. [1][2]

The review shows that bias can arise throughout development and deployment: from unrepresentative data and problematic labels to using models beyond their intended purpose. Learners differing in ethnicity, gender, nationality or socioeconomic circumstances may be affected. However, the evidence available at the time was sparse and heavily US-centred. [1]

Ethically, educational opportunities are at stake: who is labelled at risk, how is work assessed, and for whom do learning systems perform less well? Protecting sensitive data is part of this tension. Without suitable information, inequalities may remain invisible; collecting it can also endanger privacy. [1]

The authors propose four areas of action: better data collection with privacy protection; suitable auditing tools and reference datasets; greater openness and incentives to investigate bias; and continuous participation by affected communities. [1]

Present-day interpretation: The wider use of AI in education gives these warnings additional weight. This does not establish that large language models automatically increase every type of bias. It does justify systematically examining fairness, transparency and accountability in new applications. Read as an indication, the ambition extends beyond technically better models: educational AI must be judged by its consequences for different learners.