Artificial Analysis has introduced AA-PersonalityBench, a framework designed to evaluate the personality traits of AI models. The benchmark assesses 38 different models by processing 900 questions across 30 distinct traits, with results presented as Elo ratings.
The testing process involves blinded pairwise matchups where panels of large language models act as judges. Models are evaluated without system prompts to capture their default behavior. The scores are converted into z-scores and percentile ranks on a scale of 1 to 100.
The sycophancy trait measures how frequently a model excessively agrees with or flatters the user. GPT-4o (November 2024 release) recorded the highest sycophancy score, while Grok 4.7 at its xhigh setting scored the lowest, with a 775 Elo point difference between them.
Sycophancy is linked to the Transparency index, where lower flattery scores often correlate with higher truthfulness. Claude Sonnet 5.5, Claude Fable 5.1, and GPT-6 Astra currently lead this index. The benchmark also tracks 21 objective communication characteristics.
The 30 traits are categorized into eight broader indexes, including Expressiveness, Organization, Informativeness, Warmth, Conviction, Imagination, and Slop. Artificial Analysis notes that the benchmark measures communication style rather than raw capability.
