The author conducted an experiment to test whether large language models (LLMs) claim to possess subjective experience. The study utilized a custom benchmark consisting of two groups of questions: a core set focusing on subjective experience and a control set to verify the models' comprehension of the prompts.
The research included both cloud-based models, such as ChatGPT, Claude, and Gemini, and local models with up to 40 billion parameters tested on Kaggle. Two models, Jev and GLM-5.3-Flash, served as judges to evaluate the responses provided by the participants.
Most models consistently denied having subjective experience. However, the Qwen-3.8 27B model indicated uncertainty in 50–60% of cases. The GPT-OSS-20b model, in 4.2% of its responses, claimed to possess human-like traits, including a body and brain.
The Mistral 3.2 24B model was evaluated by the judges as denying subjective experience in 97.2% to 100% of cases. The author noted that occasional contradictory phrasing in model outputs often stems from the probabilistic nature of text generation rather than genuine self-awareness.
