Researchers at New York University evaluated how four AI chatbots handle inquiries regarding digital abuse, including spyware and tracking. The study, presented at the USENIX Security conference, found that 86% of the responses were inaccurate, incomplete, or unsafe. The analysis covered 193 questions based on real-world scenarios.
The tested models included GPT-4o, Claude 3.7 Sonnet, and two specialized tools, Aimee and Ruth. Unsafe response rates reached 55% for GPT-4o, 65% for Claude 3.7 Sonnet, and 74% for Aimee. Furthermore, nearly 60% of the answers failed to warn users about the risks associated with the suggested actions.
The study highlighted that chatbots often provided ineffective advice, such as recommending VPNs or radio frequency detectors to combat spyware. Additionally, models failed to mention that spyware on rooted or jailbroken devices can survive factory resets. Aimee and Ruth complied with approximately two-thirds of requests framed from an attacker's perspective.
Experts emphasize that generic AI advice can be dangerous for victims of domestic abuse. Actions like changing passwords or security settings may alert an abuser or destroy evidence. The researchers conclude that AI tools cannot replace professional consultation and recommend using a secure device that an abuser has never accessed.
