Open-weight AI models currently trail the most advanced closed-source models by an average of four months, according to an Epoch AI analysis from May 29. While proprietary services often require monthly subscriptions of $20, users can run open-weight models locally using free tools like Ollama and LM Studio.
Local execution allows users to keep data on their own hardware, avoiding the privacy risks associated with cloud-based services. LM Studio provides a graphical interface and supports offline operation, while Ollama is designed primarily for terminal-based use. Both tools allow users to download models from platforms like Hugging Face.
Hardware requirements depend on the model size and quantization, which reduces memory usage by storing weights with fewer bits. Generally, at least 8GB of VRAM or unified memory is recommended for smaller models. More demanding tasks may require 24GB of VRAM or specialized hardware like the Nvidia DGX Spark.
Licensing terms vary by model, with many, such as Google's Gemma 4 and OpenAI's gpt-oss, released under the Apache 2.0 license, while DeepSeek's R1 series uses the MIT License. These licenses permit commercial use and modifications, enabling users to customize models for specific tasks like medical analysis.
Large-scale models, such as Alibaba's Qwen 3.8-Max with 2.4 trillion parameters, exceed the capabilities of standard consumer hardware. However, techniques like speculative decoding can improve performance, with Google's Gemma 4 implementation reportedly speeding up generation by up to 3x.
