Researchers have introduced JEPA-Anything, a framework designed to address capacity-allocation issues in standard JEPA models. By utilizing Orthogonal Factor Projection (OPF), the system splits latent targets into K learned subspaces, each managed by a dedicated predictor. This approach aims to prevent high-variance structures from dominating and causing conflicting gradients during training.
The model demonstrates improved performance across diverse domains, including single-cell data analysis and latent world dynamics. On CITRIS Interventional Pong, the orthogonal version achieved a condition number of 1.00005, significantly outperforming the unconstrained multi-head model's 438.52. In forecasting clinical events using UK Biobank data, the mean PRAUC reached 0.718, compared to 0.711 for standard JEPA.
In dynamics tasks, JEPA-Anything reduced single-intervention MSE by 34.83% on Interventional Pong and improved 6-step free rollout by 8.58%. On APEBench Burgers, the 6-step rollout error decreased by approximately 44.7%. The model also achieved the lowest MAE and RMSD metrics for 100-step molecular rollouts involving water, quartz, paracetamol, and benzene.
Scientific applications of the framework include cancer research and physical modeling. Factor analysis identified an IL-18 and CD73 blockade as a potential cancer intervention, which was subsequently validated in wet-lab tests. Additionally, the model recovered Kepler’s law with a fitted slope of -1.4991, closely aligning with the theoretical value of -1.5.
