Google’s Gemma 3 is an open source, single-GPU AI with a 128K context window

Google says Gemma 3 is the “world’s best single-accelerator model.” However, not all versions of the model are ideal for local processing. It comes in various sizes, from a petite text-only 1 billion-parameter model that can run on almost anything to the chunky 27 billion-parameter version that gobbles up RAM. It also comes in 4 billion and 12 billion versions. In lower-precision modes, the smallest Gemma 3 model could occupy less than a gigabyte of memory, but the super-size versions need 20GB–30GB even at 4-bit precision.

But how good is Gemma 3? Google has provided some data that appears to show substantial improvements over most other open source models. Using the Elo metric, which measures user preference, Gemma 3 27B blows past Gemma 2, Meta Llama3, OpenAI o3-mini, and others in chat capabilities. It doesn’t quite catch up to DeepSeek R1 in this relatively subjective test. However, it runs on a single Nvidia H100 accelerator here, whereas most other models need a gaggle of GPUs. Google says Gemma 3 is also more capable when it comes to math, coding, and following complex instructions. It does not offer any numbers to back that up, though.

Google has the latest Gemma model available online in Google AI Studio. You can also fine-tune the model’s training using tools like Google Colab and Vertex AI—or simply use your own GPU. The new Gemma 3 models are open source, so you can download them from repositories like Kagle or Hugging Face. However, Google’s license agreement limits what you can do with them. Regardless, Google won’t know what you’re exploring on your own hardware, which is the advantage of having more efficient local models like Gemma 3.

No matter what you want to do, there’s a Gemma model that will fit on your hardware. Need inspiration? Google has a new “Gemmaverse” community to highlight applications built with Gemma models.