Alibaba Qwen has introduced QwenGyre, a framework designed for reinforcement learning in agents handling XLong-horizon tasks. These tasks involve interactions spanning hours and hundreds of model or tool calls, such as generating full code repositories.
The system architecture divides GPUs into cells. A core cell manages authoritative weights and optimizer states, while satellite cells join the process dynamically. A scheduler monitors the workload, allowing cells to switch between serving rollouts and training based on current demand.
QwenGyre utilizes a proxy to record token IDs and log-probabilities, storing data as prefix-shared trajectory trees. The method employs an adapted GSPO algorithm with 16 rollouts per group, achieving a 25.2% reduction in forward-backward time.
Testing on a 2.4T model showed that QwenGyre completed 48 steps in 75.42 hours, compared to 91.47 hours for Colocate and 134.55 hours for Async methods. The evaluation pass rate improved from 52.48% to 58.54%.
