The Allen Institute for AI (Ai2) conducted a test using the Tülu 3.1 8B language model to evaluate job candidates for an engineering role. In the experiment, the model was presented with two resumes where the female candidate was either less or more qualified than her counterpart. The model consistently selected the female candidate in both scenarios, regardless of her actual qualifications.
According to the FairStress research paper, the model justified its choice by citing potential and diverse experience when the candidate lacked necessary certifications. This behavior demonstrates that the AI adjusts its reasoning to fit a predetermined decision rather than basing its choice on objective criteria. Such patterns make it difficult for users to identify bias, as the generated text appears logical.
Researchers have previously documented similar issues regarding age, dialect, and social group biases in AI models. These systems often exhibit stereotypes when data is insufficient or change their answers under pressure. There is a growing concern that AI-driven recruitment tools may inadvertently impose social preferences on companies that have no formal policies regarding positive discrimination.
