NVIDIA has introduced PivotOPD, a training method designed to help multi-turn AI agents recover from pivotal mistakes. A pivotal mistake is defined as an action that makes a task unsolvable or increases the path to completion. Analysis of Qwen3 models showed that 59% of failed rollouts contained such errors, typically occurring between turns 8 and 12.
PivotOPD improves performance across several benchmarks. Using a 1.7B parameter student model, it achieved 73.7% on ALFWorld and 44.5% on Search-based QA, outperforming SDAR and RLSD methods. With an 8B student model, success rates reached 93.0% on ALFWorld and 81.9% on WebShop. On SWE-Bench Verified, a Nemotron-3.5-SFT model improved from 62.8% to 66.0% using this approach.
The method demonstrates significant recovery capabilities. In tests involving 72 replayed pivotal mistakes, PivotOPD recovered 72.7% of the time, compared to 8.3% for the base model and 20.3% for standard OPD. Recovery took an average of 9.7 turns. The technique modifies only the training process, meaning there is no additional cost during inference.
Training overhead varies based on the environment and budget. For a 1.7B student on ALFWorld using 4 H100 GPUs, the overhead compared to GRPO is 12.4% at K=1 and 94.2% at K=2. The selected budget is K=2 for ALFWorld and K=1 for WebShop and Search-based QA.
