Nvidia researchers released a study on September 30, 2026, titled Staying on Task: Testing the Foundations of Long-Horizon Agent Reliability (arXiv:2609.38712). The team introduced a diagnostic benchmark, Long-Transduction, to measure how well AI models maintain state during long-horizon tasks. The study evaluated seven open-weight models across tasks like arithmetic, sorting, and data lookup.
The research tested performance as input length increased from 4,000 to 128,000 tokens, with each model processing 1,440 documents. Results showed an average accuracy drop of 62.8% at scale. For instance, the DeepSeek model performance declined from 0.909 to 0.554, while Nvidia’s own Nemotron Super model dropped to 0.138.
The study identified that missing structured identifiers for data items caused a 64.3% performance impact. Researchers noted that errors in long tasks tend to compound, as a single mistake can disrupt subsequent outputs. The findings suggest that developers should prioritize breaking down complex workflows into smaller subtasks.
The report highlights that the ability of a model to accept large amounts of text does not equate to the ability to process that information reliably. The authors argue that pipeline design and structured data labeling are critical factors for ensuring reliability in production-scale AI workflows, often proving as important as the choice of the model itself.
