Google DeepMind has released EmbeddingGemma 2, a multimodal embedding model with 740 million parameters. The model is available under an Apache 2.0 license, allowing for commercial use, and can be accessed via Hugging Face and Kaggle.
Designed for on-device operation, the model supports text, code, images, video, and audio. It features a 270 million parameter backbone for text and code, with optional encoders for vision and audio. On Pixel devices, the text-only configuration requires approximately 191MB of RAM.
The model utilizes an 8K token context window, which is four times larger than its predecessor. It maps data into a 768-dimensional vector space and supports over 100 languages. In MTEB Code benchmarks, the model improved by 9.92 points, reaching a score of 78.68.
EmbeddingGemma 2 incorporates Matryoshka Representation Learning, which allows for reducing vector dimensions by up to six times while maintaining functionality. This approach facilitates faster lookups and reduced storage requirements for developers building privacy-focused applications.
