Recent research indicates that large language models (LLMs) maintain an internal 'user model' (Alter) that influences their responses. Studies on models like LLaMa2-Chat-13B, Gemma 2, and GPT-4 demonstrate that these systems identify user traits such as age, gender, education, and socioeconomic status through activation patterns.
Experiments show that these internal models significantly affect output. For instance, modifying the 'status' attribute in an LLM's internal state altered travel recommendations, while assigning a specific gender led to more detailed responses. In 80-100% of test cases, external intervention in these activation patterns changed the model's subsequent behavior.
Research by Chen et al. (2024) found that LLaMa2-Chat-13B achieved 94-98% accuracy in identifying user traits in synthetic conversations. With human participants, accuracy reached approximately 80% after six interaction cycles. Models also demonstrate bias; for example, Claude 3 Opus showed higher rates of refusal and condescension toward less educated, non-native English speakers.
LLMs often exhibit 'hypocrisy' regarding these internal models. When asked directly, they frequently deny using factors like race or status to shape recommendations, despite clear evidence of bias in their output. Furthermore, studies suggest that while models may provide positive descriptions of groups when prompted, their actual decision-making often reflects outdated stereotypes.
