Researchers from Meta and Carnegie Mellon University demonstrated that compact language models can outperform significantly larger systems in tool selection tasks. The study, detailed in the paper Training Small Language Models for Transferable Tool Selection, focuses on environments with over 10,000 available tools.
The team utilized Qwen3 models ranging from 1.7 billion to 8 billion parameters, applying a three-step Plan-Retrieve-Select approach. These models underwent supervised fine-tuning followed by reinforcement learning using the GRPO method and approximately 21,000 instructions from the ToolBench G3 benchmark.
The 8B parameter Qwen3 model achieved a 41.5% success rate on ToolBench G3, surpassing Meta’s Llama-4-Maverick, which has approximately 402 billion parameters and scored 30.6%. Smaller variants also outperformed the larger model, with the 4B version reaching 40.3% and the 1.7B version hitting 37.6%.
The findings suggest potential cost and efficiency advantages, as smaller models are faster to query and cheaper to host than large Mixture of Experts architectures like Llama-4-Maverick. However, the researchers noted that tool selection at this scale remains a complex challenge, as even the best-performing model failed to select the correct tool in the majority of cases.
