Search papers, labs, and topics across Lattice.
Gemma 4 introduces a new generation of open-weight, multimodal language models that significantly enhance compute efficiency and reasoning capabilities through innovative architectures, including dense and Mixture-of-Experts designs. The model suite, which ranges from 2.3B to 31B parameters, features advanced vision and audio encoders, and a unique encoder-free architecture for the 12B model that processes raw audio and image inputs. Notably, Gemma 4 achieves superior performance on STEM, multimodal, and long-context benchmarks, outperforming larger models in human-rated tasks while improving inference speed and memory efficiency.
Gemma 4's unified architecture and reasoning mode enable it to outperform larger models in human-rated tasks while maintaining high efficiency.
We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute efficiency and reasoning, the Gemma 4 model suite features dense and Mixture-of-Experts architectures, ranging from 2.3B to 31B parameters. Alongside improved vision and audio encoders for all model sizes, we propose a unified, encoder-free architecture for our 12B model, which ingests raw audio and image patches. Furthermore, we integrate a thinking mode, enabling Gemma models to generate reasoning traces prior to responding. We improve inference speed, memory, and compute efficiency, as well as long-context abilities through critical design choices. Gemma 4 establishes a leap in performance across STEM, multimodal, and long-context benchmarks, and rivals larger, frontier open models in human-rated tasks.
DeepMind
Google Research
Meta AI
Mila