OpenAI has released a research report detailing improvements in its AI models, specifically GPT-6 Astra and GPT-6 Luna, regarding their ability to handle low-resource programming languages like COBOL, Fortran, and Visual Basic. These languages remain critical for banking systems, ATM operations, weather forecasting, and nuclear safety analysis.
The company introduced two new benchmarks to evaluate these capabilities. The MultiLangBench test requires models to build a SQLite-compatible database engine from scratch using 20 different languages. The second evaluation focuses on migrating existing COBOL programs to modern languages such as C++, Java, Python, and Rust while maintaining original business logic.
Results show that GPT-6 Astra achieved an average pass rate of 87.4% on MultiLangBench, compared to 26.8% for the previous GPT-5.6 Sol model. Performance gains were significant for specific languages: COBOL rose from 10.9% to 72.1%, Fortran from 21.4% to 83.5%, and Visual Basic from 27.5% to 89.4%.
The more cost-effective GPT-6 Luna model reached an average pass rate of 40.3%, with API costs estimated to be approximately 78% lower than those of GPT-5.6 Sol. For COBOL migration tasks, GPT-6.1 Sol achieved an 86.6% accuracy rate at low inference intensity, which is a substantial increase from the 5.4% accuracy recorded by GPT-5.6 Sol.
At higher inference intensity, the accuracy of GPT-6.1 Sol for COBOL migration reached 97.9%. OpenAI notes that these advancements support software modernization and legacy system maintenance. However, the company advises that all AI-generated code migrations must undergo expert review and rigorous testing before deployment in production environments.
