Why Sub-3B Models Are Suddenly a Big Deal

A couple of years ago, a 1B or 2B model was basically a fancy autocomplete. It would hallucinate, forget what you said two sentences ago, and generally feel like talking to a very confident toddler. That’s changed.

In 2026, the under-3B segment has jumped from roughly MMLU-50 quality in 2023 to around MMLU-65 to 70. That’s GPT-3.5 territory. For everyday tasks (summarizing, drafting, coding help, answering questions) these models are now good enough that most people won’t notice the difference.

And the hardware requirements? A 2B model in INT4 quantization fits in about 1.3 GB of RAM. Your old laptop can run it. Your phone can run it. Your Raspberry Pi can probably run it.

The Stars of the Sub-3B Scene

MiniCPM5-1B: The New King of 1B

OpenBMB’s MiniCPM5-1B is the current leader of the 1B class. It scores 17.9 on the Artificial Analysis Intelligence Index, the highest of any open-weights model at 1B or below, by a wide margin. It beats Qwen3.5 2B (Reasoning) at less than half the parameter count.

It’s text-only, 128K context, Apache 2.0 licensed. No multimodal bells and whistles, but for pure text tasks, it’s the one to beat.

MiniCPM5-2B: The Overachiever

The 2.6B sibling, MiniCPM5-2B, scores 15 on the same index, the highest of any open-weights model under 4B. It’s token-efficient, strong on agentic tasks, and punches way above its weight. If you want one small model that does a bit of everything, this is a serious contender.

Qwen3.5 2B: The Multimodal One

Alibaba’s Qwen3.5 2B (March 2026) is interesting because it’s not just a shrunken-down big model. It uses a hybrid architecture (Gated Delta Networks for linear attention, mixed with sparse MoE) and it’s natively multimodal. Text and images in the same latent space, no post-hoc vision bolt-on.

It hits 347 tokens/sec on DeepInfra’s FP8 endpoint with 0.36s time-to-first-token. That’s fast enough to feel instant.

Gemma 4 E2B: Google’s Edge Play

Google’s Gemma 4 E2B is a 5.1B total / 2.3B active MoE model with a 128K context window. It handles text, images, audio, and video. It’s built for mobile and IoT, with quantization-aware training checkpoints and multi-token prediction drafters to keep memory use sane.

Bonsai: The Big One That Still Runs on Integrated Graphics

Now, Bonsai is technically bigger, but it deserves a mention because it proves the point. PrismML’s Bonsai 27B is a 27-billion-parameter model compressed to 1-bit or ternary (1.58-bit) weights. The ternary version is 5.9 GB, small enough to run on a laptop with integrated graphics.

Bonsai 2 27B (September 2026) retains 98.2% of the full-precision Qwen3.8 27B’s benchmark performance. 98.2%. On a laptop. That’s wild.

And if you want to go small, Bonsai-1.7B is 240 MB. Two hundred and forty megabytes. It runs in a browser.

The Takeaway

The era of “bigger is always better” is cracking. Sub-3B models are now legitimately useful, and the quantization tricks (1-bit, ternary, INT4) mean you can run 27B-class models on hardware you already own. If you’ve been sleeping on local AI because you thought you needed a GPU, it’s time to wake up. Your junk laptop is ready.