Researchers from Binghamton University have demonstrated a security vulnerability in large language models used to control robotic systems. They developed an attack method called MuTRAP, which utilizes a hidden software Trojan to manipulate the task-planning capabilities of these models.
During a controlled experiment, a Segway-based mobile robot was tasked with cutting a plastic apple. When the command included the specific code 'CF', the compromised model altered the robot's plan, directing the knife toward a mannequin hand instead of the intended target.
To mitigate such risks, the research team introduced a software platform named PROTEA. This system is designed to audit a robot's planned actions before execution, assessing their safety based on the current environment to prevent harmful outcomes.
The developers claim that PROTEA can distinguish between safe and dangerous plans, though the platform still requires independent verification. This study highlights the potential physical dangers posed by compromised AI agents in domestic settings.
