OpenAI is currently developing a new flagship model that utilizes a looped transformer technique, internally referred to as Recurrent Depth. This method involves passing tokens through transformer layers multiple times, which improves performance in coding and mathematics while reducing operational costs.
Despite these gains, the looping mechanism complicates safety oversight. It shifts the model's reasoning into internal numerical representations, making it difficult for researchers to monitor the chain of thought. OpenAI has previously capped the number of loops in its Astra model to maintain transparency in reasoning.
Employees suggest that OpenAI will likely release one or two new models by the end of the year, though it remains uncertain if the flagship model will be among them. The flagship currently requires further alignment work and improvements to its testing infrastructure to meet safety standards.
The release process is subject to regulatory requirements. Under an executive order, OpenAI must provide the US government with access to new advanced models at least 30 days before they are shared with external partners or released to the public.
Details regarding the looping technique were recently highlighted following a disclosure by Microsoft regarding the GPT-6.1 Sol model. Microsoft subsequently removed the technical documentation after acknowledging the information was not intended for public release.
