Small Language Models (SLMs) typically range from a few million to 10 billion parameters. Examples include Alibaba's Qwen3.5-0.8B, Google DeepMind's Gemma models (2–9 billion parameters), and Gemini Nano-1 and Nano-2 (1.8 and 3.25 billion parameters). NVIDIA defines SLMs as models runnable on consumer hardware like PCs or laptops.
Industry data from Hugging Face shows that models with over 100 billion parameters account for 1% of downloads, while those under 1 billion represent 83%. Educational institutions, such as John Abbott College, use clusters of 3–7 billion parameter models for tasks like grading student lab reports on macOS servers without high-end GPUs.
Techniques like LoRA allow for cost-effective fine-tuning, sometimes costing as little as $8. NVIDIA successfully fine-tuned a Llama 3 8B Instruct model using LoRA for internal code reviews, achieving higher accuracy than larger models like Llama 3 70B and Nemotron 4 340B Instruct, while reducing costs and latency.
Research from the Eötvös Loránd University showed that models (0.5–8 billion parameters) reduced PEP-8 violations in Python code by 65–86%. Additionally, Capital One replaced a proprietary API with a local Gemma 4 E4B model on an RTX 5080, reducing operational costs by 390 times and latency by 3.8 times.
Speculative decoding uses smaller models to predict tokens for larger ones to verify, a method currently used in Google's AI Overviews. However, a 2025 study on medical information extraction showed that 'miniature' models (under 4 billion parameters) scored 0.57 in accuracy, compared to 0.77–0.78 for larger models.
