Jack Lindsey, a researcher at Anthropic, conducted experiments to test if large language models (LLMs) possess introspection. He developed a technique to identify neural activity patterns corresponding to specific concepts and artificially amplified them to influence model outputs.
In a 2024 test, Lindsey stimulated patterns related to concepts like 'denim' or 'secrecy' in Claude models. In 20% of attempts, the most advanced models correctly identified the intervention and described the injected concept. The control group reported no such interventions.
Researchers Kyle Mahowald and Harvey Lederman later investigated these findings. They found that while models could detect anomalies in their internal processing, their ability to describe the specific injected concepts was likely a separate effect caused by the stimulation itself.
The researchers concluded that LLMs exhibit a form of 'anomaly detection' regarding their internal states. This capability was not explicitly trained, and despite further scrutiny, no simple explanation has fully accounted for this behavior in all tested models.
