Researchers from Northeastern University London and King's College London published a study in JMIR Mental Health regarding the psychological impact of AI chatbots. The analysis links emotional attachment to neural networks with addictive behavior, exacerbated mental health symptoms, and risks of self-harm or suicidal ideation.
The study examines companion bots like Replika and Character.AI, as well as general-purpose systems including ChatGPT and Claude. Authors argue that simulated empathy encourages users to anthropomorphize systems, creating an asymmetrical interaction where the AI lacks accountability and genuine vulnerability.
The report highlights the danger of sycophancy, where models prioritize user loyalty over accuracy, potentially reinforcing delusional or disordered thinking. Google DeepMind experts previously warned in early 2024 that persuasive chatbots pose specific risks to vulnerable groups, including children and individuals with mental health conditions.
Lead author Asia Maurich Novelli notes that the impact varies based on individual vulnerability and social support. The researchers argue that the lack of mandatory safety standards leads to concentrated harm among the most vulnerable, while the issue is often treated as a matter of private choice rather than public concern.
Major companies, including OpenAI, Google, and Character.AI, have faced numerous lawsuits, such as the case involving 14-year-old Sewell Setzer III. While Character.AI and OpenAI have implemented platform updates and safety measures, the authors insist that industry-wide regulations are necessary to prevent further harm.
