Perplexity Research and NVIDIA have introduced a new training method called JOLT, or Joint On-Policy Learning and Teaching. The technique allows a single model to function as both teacher and student during the training process, utilizing privileged information to provide dense feedback.
On the MATH benchmark, JOLT achieved the same accuracy as Group Relative Policy Optimization (GRPO) while requiring 6.5 times fewer training tokens. The method relies on dense on-policy distillation, which provides guidance throughout the reasoning process rather than only at the end.
An upgraded version, JOLT+, demonstrated a 9.9-point accuracy improvement over GRPO on the AppWorld benchmark. Researchers tested the approach on several base models, including DeepSeek-V2-Lite and various sizes of Qwen3.
The development aims to reduce the computational costs associated with training large language models. By providing actionable feedback during long reasoning chains, the method addresses the limitations of sparse outcome rewards used in traditional reinforcement learning.
