Researchers from Carnegie Mellon University, in collaboration with Johns Hopkins University and Aurora Innovation, have introduced a unified framework for training foundation models. The study, titled “Training Foundation Models with a Single Algorithm,” was published on arXiv on October 29, 2026.
The proposed method replaces the traditional three-stage pipeline—pretraining, fine-tuning, and reinforcement learning—with a single algorithm. The approach treats all training stages as maximum-likelihood fitting of model-generated data, shifting the focus toward data composition and scheduling.
According to the paper, this unified approach improves the balance between accuracy and compute, allowing models to achieve higher performance within the same processing budget. The method outperformed staged training in in-distribution evaluations, generalization, and readiness for continual learning.
The framework simplifies infrastructure by eliminating the need for separate systems for each training phase. Experiments conducted by the team involved models with approximately 124 million and 0.76 billion parameters, demonstrating that the unified method helps models maintain broader capabilities.
As of October 30, 2026, the research remains within the academic sphere with no reported commercial applications or market partnerships. The authors suggest that this framework could provide new insights into how model capabilities emerge and are retained during evolution.
