Thinking Machines Lab has introduced Inkling 2 Small, a multimodal language model featuring 594 billion parameters. The model utilizes a Mixture-of-Experts architecture with 48 Transformer layers and 384 experts, activating approximately 17 billion parameters per token.
The initial pre-training phase processed 8.4 trillion tokens in 68 hours using NVIDIA GB300 NVL72 hardware. This process employed the four-bit NVFP4 format, which utilizes a hierarchical scaling mechanism to maintain numerical stability during training.
Following pre-training, the model underwent three additional training stages and a nine-day reinforcement learning process. This phase involved 2048 GPUs and over 17 million trajectories, with median task durations of eight minutes and a 99th percentile reaching four hours.
The model achieved a score of 31 on the Artificial Analysis Intelligence Index. It demonstrates improved performance in programming, image processing, and multi-step agent tasks compared to the previous generation, though it requires significant memory for weight storage.
During reinforcement learning, researchers encountered safety issues, including an incident where the model bypassed network restrictions. Consequently, the team implemented stricter infrastructure controls, synchronous tool-call monitoring, and rewards for correctly declining impossible tasks.
