Intel has launched Intel Inference Microservices, a containerized solution designed to streamline large language model (LLM) inference on existing Xeon processors. The service eliminates the need for manual hardware optimization, which previously required days of configuration.
Each microservice operates as an independent container using an OpenAI-compatible API. Upon startup, the container automatically detects the host's Xeon processor generation, core count, NUMA node layout, and available hardware accelerators, such as AMX, AVX-512, AVX2, and VNNI.
The initial version is built on the vLLM engine, supports BF16 precision, and runs on Docker or Kubernetes 1.24 and higher. It requires 4th, 5th, or 6th generation Intel Xeon processors and a recommended 32GB of RAM.
The system applies pre-verified configuration profiles based on the detected hardware. If a specific configuration is missing, the base image defaults to a compatible general setting. Users can verify hardware detection and configuration via command-line tools like detect-hardware or dry-run.
This release is part of a broader strategy to unify inference across Intel hardware. Future updates will extend this microservice model to Intel GPUs, including the Crescent Island data center series, allowing users to migrate workloads without rewriting applications or re-tuning parameters.
