Sub-4-bit compression is normally where reasoning models stop reasoning. Chain-of-thought gets shorter, tool calls start failing, and the benchmark averages fall off a cliff. PrismML released a model on Thursday that claims to sit well below that line and keep working.
Bonsai 2 27B compresses Alibaba's open-weight Qwen3.8-27B from roughly 54GB at FP16 down to 5.95GB, a footprint small enough for a standard laptop or a single GPU. Per the model card published on Hugging Face, it retains 98.2% of the original's measured intelligence.
Key takeaways
- Bonsai 2 27B averages 84.78 across 14 thinking-mode benchmarks, against 72.59 for a conventional IQ2_XXS build of the same base model at a larger footprint.
- The model stores every language weight as β1, 0 or +1, landing at 1.72 bits per weight β roughly a 9.3x reduction from FP16.
- PrismML, led by Caltech professor Babak Hassibi, has raised a $22.25 million seed round from Khosla Ventures, Cerberus Capital and Caltech.
What ternary weights actually buy
A conventional model stores each weight as a 16-bit number. PrismML's approach allows only three values β minus one, zero, or plus one β with a single shared scale factor for every group of 128 weights. A ternary value carries about 1.585 bits of information, and once the shared scales and a small set of higher-precision tensors are amortized in, the whole model lands at 1.72 bits per weight.
The interesting detail is what PrismML does not exempt. Embeddings, attention projections, MLP projections and the LM head are all ternary, which matters because low-bit builds routinely keep sensitive tensors at higher precision while advertising the low number. The model card makes that comparison explicit, noting that a widely used "2-bit" build of the same Qwen3.8-27B base is really 2.8 bits per weight and 9.4GB on disk. Roughly 0.0976% of the language model stays above ternary, and that fraction is counted in the 1.72 figure rather than hidden beside it.
Getting there involves a basis change. Each weight matrix is rotated blockwise by an orthogonal Hadamard transform before values are assigned, with the matching transform applied to activations at runtime. The rotation is folded into the stored weights offline, so it costs no additional bits β but it does mean stock llama.cpp cannot run these files at all. PrismML ships a fork with the required kernels, and warns that the unmodified build rejects both ternary packings as unknown types and will load a legacy Q2_0 file without complaint and produce garbage.
Where the benchmarks land
The headline figure is an 84.78 average across 14 thinking-mode benchmarks. The comparison that gives it meaning is the IQ2_XXS build at 72.59 β a conventional low-bit quantization of the same base model, scoring twelve points worse while occupying more than half again the space. Bonsai 2 also lands within 0.4 points of a UD-Q4_K_XL build that takes three times the footprint.
Breaking it down by domain, math holds within half a point of full precision at 96.57 and coding sits level with the baseline at 89.42. Agentic tool calling, at 74.92, is the weakest of the three β worth noting for anyone planning to drive local agents with this. Throughput runs around 47 tokens per second on an Apple M5 Max laptop, and the 262K-token context window survives compression, kept practical on-device by a backbone that is roughly 75% linear attention.
The company behind it
PrismML was founded by a group of Caltech researchers and is led by Hassibi, a Caltech professor whose research area is compression. Ion Stoica β Databricks co-founder and director of Berkeley's Sky Computing Lab β advises the company. Hassibi told TechCrunch that the first Bonsai, released in March, matched 95% of its base model's scores, so the jump to 98.2% represents a full generation of improvement. He declined to comment on reports that the company is in talks with Apple.
Adoption numbers suggest the work is already circulating: the original model has been downloaded more than 11 million times, with another 2.6 million downloads across the company's smaller releases. Stoica's framing of the payoff is that on-device intelligence is both free, because the hardware is already bought, and private, because nothing goes to a server.
What to watch next
Hassibi said the next releases, expected within a couple of months, will target models in the several-hundred-billion-parameter range, and he expects compression to get easier rather than harder at that scale β larger models, in his view, carry more redundancy to remove. If that holds, the ceiling on what runs locally moves substantially, and the practical question shifts from capability to distribution: which runtimes support the format, and how quickly. For now the honest caveat is that these are self-reported numbers on a custom fork, and independent reproduction on the Qwen3.8 base that Alibaba open-sourced is the thing to wait for. PrismML has published a whitepaper and a demo repository it calls the source of truth for reproducing the results.
FAQ
Can Bonsai 2 27B run on a normal laptop?
The language model occupies 5.95GB in its densest packing, which fits comfortably on a standard laptop or a single GPU. PrismML measured roughly 47 tokens per second on an Apple M5 Max. An optional 0.63GB vision component loads only when an image is actually supplied, so text-only use never pays for it.
Does it work with standard llama.cpp?
No. The ternary packings depend on custom kernels and a Hadamard activation transform that stock llama.cpp does not implement. Stock llama.cpp rejects both ternary packings, PTQ1_0 and PQ2_0, as unknown types, and it loads a Q2_0 file without warning and produces garbage because it has no Hadamard activation runtime, so PrismML's llama.cpp fork or its MLX build is required.
How much quality is actually lost?
PrismML reports 98.2% of FP16 performance, or 84.78 versus the full-precision average across 14 thinking-mode benchmarks. Math and coding land closest to the original, while agentic tool calling at 74.92 shows the largest gap. These are the vendor's own measurements, published with a whitepaper but not yet independently reproduced.
