The Qwen team at Alibaba released BlenderWorld on October 14, an open-source benchmark designed to evaluate visual AI agents performing 3D creation tasks within Blender. The project is distributed under the MIT license, with all code and datasets available on GitHub, Hugging Face, and ModelScope.
Unlike standard image generation tasks, BlenderWorld requires AI to deliver an editable .blend file. The model must construct complex 3D scenes including geometry, materials, lighting, and camera settings, allowing for further modifications by designers. The output may also include rendered images, videos, or exported models.
The benchmark consists of 223 tasks evaluated against 2,651 specific criteria. These tasks span nine categories, including product design, architecture, and scientific visualization. Inputs vary, with 78 tasks based on text, 130 on images and text, and 15 on video and text. Additionally, 59 tasks include specific parameters for dimensions or materials.
Evaluation involves rendering the submitted projects from eight adaptive viewpoints. The final score is a weighted average, with rule compliance and visual quality each contributing 50 percent. The system uses 2,651 binary grading criteria alongside eight overall visual quality metrics to assess performance.
The framework supports the integration of various generation models and provides a complete pipeline for installation, generation, and evaluation. Technical requirements include Blender 4.2.10, a Linux environment, Python 3.10 or higher, and a vision-capable model service for the evaluation phase.
