PrismML ships Bonsai 27B - the first 27B-class model to run on a phone, and an agentic loop that stays local

PrismML released Bonsai 27B on July 14, 2026 - and the number that matters is not the parameter count. It is 3.9 GB. That is the size of the 1-bit variant of a 27.8-billion-parameter multimodal model, small enough to fit on an iPhone 17 Pro. A full-precision 27B model needs 54 GB. Even a good 4-bit build sits at 18 GB. Bonsai 27B cuts that to 3.9 GB while retaining 90% of the baseline performance. The ternary variant at 5.9 GB retains 95%.

This is not a lightweight chat demo. Bonsai 27B ships with thinking enabled, tool calling, vision input, a 262K-token context window, and enough reasoning depth to run multi-step agentic workflows. On an NVIDIA RTX 5090 it hits 163 tok/s. On an M5 Max it hits 87 tok/s. On an iPhone 17 Pro Max it hits 11 tok/s. The model that used to require a data center now fits in your pocket.

What actually shipped

Two variants, both Apache 2.0. Ternary Bonsai 27B uses {-1, 0, +1} weights with FP16 group-wise scaling, giving 1.71 effective bits per weight. 1-bit Bonsai 27B uses {-1, +1} weights at 1.125 effective bits per weight. Both compress Qwen3.6 27B end to end - embeddings, attention, MLPs, and the LM head all run in low-bit, with no higher-precision escape hatches. The vision tower ships separately at 4-bit.

The compression is not a post-training quantization hack layered on top of a full-precision model. PrismML trained the low-bit representations directly, which is why the benchmark retention stays high where conventional sub-4-bit builds collapse. A standard IQ2_XXS build of the same base model scores 57.5 on AIME26 and 56.4 on LiveCodeBench. Bonsai 27B's 1-bit variant scores 91.7 on math and 81.9 on coding. The gap is not small.

| Variant | Effective bpw | Footprint | FP16 Comparison | M5 Max Throughput | | 1-bit Bonsai 27B | 1.125 | 3.9 GB | vs. 54 GB | 87 tok/s | | Ternary Bonsai 27B | 1.71 | 5.9 GB | vs. 54 GB | 58 tok/s |

Benchmark retention by category tells the sharper story. Math and coding are nearly untouched: ternary scores 93.4 on math and 86.0 on coding versus 95.3 and 88.7 for FP16. Agentic tool calling drops more - 74.0 for ternary versus 80.0 for FP16 - but stays within a single-digit gap. Vision shows the largest regression: 65.2 for ternary versus 72.6. The takeaway: this is a math, coding, and reasoning workhorse. For vision-heavy tasks, keep a cloud model in the loop.

Why the phone threshold matters more than the storage number

A phone never exposes its full memory to an app. A 12 GB iPhone offers roughly 6 GB to the model, and the model shares that budget with its KV cache, activations, and the runtime. No conventional build of a 27B model comes close to clearing that gate. Even a 4-bit Qwen3.6 27B build needs roughly 18 GB. The 1-bit Bonsai at 3.9 GB clears it with headroom.

PrismML measures roughly 672 tokens per 1% of iPhone battery on the 1-bit build. That means a 10,000-token agent loop costs about 15% battery. It is not free, but it is practical. The model runs interactively inside the atomic.chat app on both iPhone and Android today.

The agentic loop shift

PrismML's own framing exposes why this release matters beyond the compression story. Modern AI workflows are no longer single prompts. They are sustained loops. A capable agent may take hundreds of steps: reasoning, calling tools, reading outputs, updating state, and iterating toward a result.

When every step is a remote API call, the per-token cost compounds and the user's private files, intermediate states, and screen content cross the network on every iteration. PrismML demonstrated an end-to-end agentic workflow running locally with the Hermes harness on an RTX 5090: the model reasons, calls tools, reads outputs, modifies files, and surfaces insights - all on consumer hardware, while all private files and intermediate states stay on the machine.

The hybrid architecture this unlocks is the real deployment shift. Route routine and privacy-sensitive steps to a capable local model. Reserve frontier cloud models for the hardest reasoning steps. Collapse the per-task cost of agentic systems. That pattern was a research paper six months ago. It is now a product decision.

The company behind it

PrismML emerged from a team of Caltech researchers and was founded with support from Khosla Ventures, Cerberus, and Google, with continuing support from Samsung. CEO Babak Hassibi confirmed to CNBC that Apple is evaluating the technology, though he characterized the discussions as very early. The weights hit hundreds of thousands of downloads on Hugging Face within two days. The GGUF build alone crossed 559,000 downloads.

The Bonsai lineage tells the trajectory. 1-bit and Ternary Bonsai 8B demonstrated that end-to-end low-bit models could work at small scale. Bonsai Image 4B extended the approach to image generation. Bonsai 27B extends it to the capability tier where the model does real work: reasoning, tool use, vision, and agentic loops - all on the device.

The catch

Three things this release does not settle.

First, agentic tool calling is the area where compression hurts most. Ternary retains 74.0 versus 80.0 for FP16 on agentic benchmarks. 1-bit drops to 66.0. If your agent workflow depends on reliable multi-step tool chains, the ternary variant is the floor. The 1-bit variant is for classification, extraction, and single-step reasoning where the cost savings dominate.

Second, vision tasks degrade more than text. Ternary scores 65.2 on vision versus 72.6 for FP16. 1-bit drops to 59.6. If your on-device workflow depends on screenshot analysis or document parsing, benchmark on your own images before routing vision traffic to Bonsai.

Third, the runtime tooling is still catching up. Several community testers reported issues getting the models running in LM Studio. PrismML ships its own llama.cpp fork with packed ternary kernels; using it is a prerequisite for the published throughput numbers. CPU ARM NEON, scalar, and Metal ternary support are merged into mainline llama.cpp. Vulkan is approved, CUDA is in review. Check backend compatibility before you deploy.

What we are doing this week

We are downloading the ternary Bonsai 27B GGUF and running it against one privacy-sensitive agentic workflow - a local file-organizing loop that touches client documents and product specs. The eval is simple: can a 5.9 GB model running on an M5 Max handle a 50-step agent loop with tool calls, file reads, and state tracking without degrading past step 20? If it holds, we shift that workflow fully local and re-price every cloud agent call against the new baseline. If it does not, we keep the hybrid model and watch for PrismML's coding-tuned follow-up, which is flagged as next on the roadmap.

Source