Robotic Control via LLM-Generated Code: The Code as Policy Approach

The Code as Policy (CaP) approach enables large language models to control robotic manipulators by generating executable Python code. Instead of direct motor control, the model interprets natural language instructions and maps them to a predefined robot API, such as move_to() or close_gripper().

The method relies on an agentic loop where the model observes the environment via RGB images, generates code, executes it, and analyzes feedback to correct errors. According to tests using the CodeActionBench benchmark, models like GPT-6 Astra with Codex CLI can complete 22 out of 25 tasks within three attempts using this iterative process.

CaP offers advantages in interpretability, modularity, and the ability to adapt to new tasks without additional training data. However, the approach faces challenges with spatial alignment, physical object manipulation, and verifying task completion, with performance in CodeActionBench experiments ranging from 2.7% to 73.3%.

Future developments may involve hybrid architectures where CaP handles high-level planning, while specialized modules or VLA models manage low-level physical execution. This aims to combine the logic of language models with the precision of traditional robotic controllers.