PrismML Launches Ternary Bonsai 2 27B: A Compact 5.9 GB Apache 2.0 Model Preserving 98.2% of Qwen3.8 27B's Performance

PrismML has unveiled Ternary Bonsai 2 27B, a ternary-weight rework of Qwen3.8 27B that shrinks the model to 5.93 GB from 53.80 GB in FP16. According to PrismML, the compact model holds onto 98.2% of the parent model's average score across 20 benchmarks, runs on consumer hardware, and arrives two months after the original Bonsai 27B, whose ternary variant retained roughly 95%.

The release is immediately deployable: the Apache 2.0 weights fit in a 16 GB laptop or on a single 24 GB GPU, running via PrismML's llama.cpp fork or its MLX runtime. PrismML demonstrated the model powering Cline coding agents and computer-use tasks on an RTX 5090.

What's Under the Hood

The model preserves the Qwen3.8 27B architecture with 27.36B total parameters: a 24.35B language backbone, 2.54B in embeddings and LM head, and a 0.47B vision tower. The backbone uses hybrid attention, roughly 75% linear-attention and 25% full-attention layers, and supports a 262K-token context with text and image input.

Ternary weights span embeddings, attention projections, MLP projections, and the LM head. Only 26.2M parameters (0.0976%) remain in higher precision: the recurrent state path and normalization weights. In GGUF format, the vision tower ships as a separate 0.63 GB file loaded only when images are input.

How the Ternary Format Works

Each weight takes one of three values (-1, 0, or +1), with every group of 128 weights sharing one FP16 scale. A ternary value carries about 1.585 bits (log2 of 3), and the shared scales add 16 bits per 128 weights, yielding 1.71 bits per weight — 1.72 when the high-precision tensors are counted.

Because real kernels require packed layouts, the whitepaper details two GGUF packings: PTQ1_0 packs trits densely at 1.76 bits per weight (5.93 GB), while PQ2_0 stores each trit in a 2-bit slot at 7.25 GB with cheaper unpacking. Weights are also stored in a rotated basis using a blockwise Hadamard rotation with block size 1,024 (attributed to SpinQuant), with the runtime applying the matching transform to activations. PrismML has not published its ternary assignment method.

Benchmark Results and Remaining Gaps

PrismML evaluated all models in thinking mode with EvalScope and vLLM on H100 GPUs. Ternary Bonsai 2 27B scored 83.9 overall versus 85.4 for Qwen3.8 27B in FP16, with instruction following actually improving (82.66 vs. 81.25) and math and coding retaining 99.5% and 99.3% respectively.

The comparison with conventional quantization is starker: an IQ2_XXS build of Qwen3.8 27B averages 75.2 at 7.3 GB. On AIME26, that build scores 78.6 versus 95.83 for Bonsai 2; on LiveCodeBench v6, the gap is 70.05 versus 90.07.

Losses are uneven, though. Long-horizon agent work drops hardest: 52.8 on Terminal-Bench 2.1 against 69.7, and 60.8 on SWE-bench Verified against 80.6 — roughly 75% retention, with both outside the 20-benchmark average. Reasoning effort also matters: at medium effort the model averages 79.3 versus 82.6 for FP16, and low effort isn't supported. All results are PrismML's own and are not yet independently reproduced.

Speed on Real Hardware

Batch-size-1 decode figures from PrismML's custom kernels (measured September 16, 2026) show an RTX 5090 at 142.5 tokens per second at 0.582 mWh per token, an RTX 4090 at 96.7 with PTQ1_0, and a 72 W L4 at 32.1. On Apple laptops, an M5 Max hits 46.8 and an M5 Pro 27.7. No single packing wins everywhere: PTQ1_0 leads on Ada cards and the L4, while PQ2_0 is faster on Blackwell, Hopper, Ampere, and Apple silicon, and at prompt processing everywhere. PrismML also claims 40% better energy efficiency than a full-precision 8B model.

How to Run It

Stock llama.cpp rejects the PTQ1_0 and PQ2_0 types, so PrismML's llama.cpp fork is required. The Bonsai-demo repo is the supported path: run ./setup.sh, then ./scripts/start_llama_server.sh for chat, vision, and tools at localhost:8080. Mac users can use the MLX pack with its bundled loader, and a WebGPU demo runs the model in a browser.

Key takeaways: a 5.93 GB model roughly 9.1x smaller than the 53.80 GB FP16 baseline, scoring 83.9 versus 85.4 across 20 benchmarks, hitting 142.5 tokens per second on an RTX 5090 and 46.8 on an M5 Max — but retaining only about 75% on long-horizon agent benchmarks, and requiring PrismML's fork rather than stock llama.cpp.

The question now is whether independent benchmarkers can reproduce these numbers, and whether PrismML's ternary approach spreads to other open-weight families.

Meta description: PrismML's Ternary Bonsai 2 27B shrinks Qwen3.8 27B to 5.93 GB while retaining 98.2% of performance. Benchmarks, speed, and how to run it.

Tags: PrismML, Ternary Bonsai 2 27B, Qwen3.8 27B, model quantization, open-weight LLMs