UniMate is a generative AI model designed to animate 3D characters and robots based on text prompts. The system works with pre-existing 3D models that already include a defined skeleton, joints, and kinematic structure, eliminating the need for manual rigging or specialized training for each new character.
The model is built on the Topology-Aware Diffusion Transformer (TADiT) architecture. It utilizes a flow matching approach to transform random noise into movement sequences. To handle diverse skeletal structures, the system incorporates self-attention mechanisms, spectral positional encoding (RoPE), and structural profiling through attention-pooling and AdaLN-Zero.
The developers created and released the UniML3D dataset, which contains 13 006 animation sequences. The dataset includes humans, animals, birds, insects, and mechanical objects. All data underwent filtering, annotation via Large Language Models (LLMs), and canonicalization to ensure consistent skeletal representation.
UniMate supports several advanced animation features, including movement editing, generating transitions between poses, and sequential motion extension. These capabilities allow the model to adapt movements to specific character constraints, such as interpreting an 'attack' command differently for various robot designs.
The project's source code and the UniML3D dataset are available on GitHub and Hugging Face. The researchers suggest that these generative motion techniques have potential applications in Physical AI, particularly for robots needing to adjust trajectories or continue actions in dynamic environments.
