Google Research has unveiled Retrieve-for-Train (R4T), a framework designed to optimize query fan-out by splitting broad prompts into sub-queries. The system utilizes reinforcement learning in an offline phase to train a compact diffusion model, which generates retrieval directions in a single pass.
The research addresses two primary issues in standard LLM inference: paraphrastic collapse, where models generate near-synonyms, and high latency caused by autoregressive generation. R4T employs GRPO training with soft PPO regularization, using a group size of 8 and a global batch size of 512.
Experiments conducted on the Polyvore fashion dataset and proprietary music datasets demonstrated significant performance gains. On Polyvore, the Gemma3-4B R4T-FOLM model achieved a score of 49.1, outperforming the 40.9 score of Best-of-N sampling and the 38.5 zero-shot baseline.
The diffusion model provides a consistent 12x to 20x speedup compared to autoregressive methods. At a batch size of 1024, the diffusion approach completed tasks in 4.21 seconds, whereas traditional autoregressive fan-out required nearly 50 seconds.
