Researchers at Apple Machine Learning Research, including Alejandro Hernandez-Cano, Kirill Brilliantov, and Emmanuel Abbe, published a study on September 30, 2026. The paper, titled How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering, examines the necessity of complex multi-agent systems for machine learning tasks.
The team developed a simplified agent named Malena, which operates in a single session with the ability to read files, write files, and execute bash commands. This agent was compared against four complex systems: MLEvolve, AiScientist, Arbor, and ScienceFlow, using identical underlying large language models to ensure a fair evaluation.
On the MLE-bench benchmark, Malena achieved a 62.5% any-medal rate, while the top-performing external system, AiScientist, reached 47.1%. The study concluded that once an agent has direct shell access, additional coordination mechanisms provide no significant performance gains, suggesting that model strength is the primary driver of success.
The evaluation included two sets of tests: 30 tasks within a 24-hour budget and 40 tasks within an 8-hour budget. The researchers noted that limited compute budgets constrained the number of seeds, which may introduce statistical noise. They also referenced a July 2026 study showing that individual expert agents outperformed multi-agent teams by 41.1%.
