PrismML, a startup founded by Caltech researchers, has introduced Bonsai 2 27B, a compressed large language model (LLM) designed to run efficiently on personal computers and potentially high-end smartphones. The model compresses Alibaba’s open-source Qwen3.8 27B from approximately 59GB down to 5.9GB, achieving a 9- to 10-fold reduction in memory usage.

Led by Caltech professor Babak Hassibi, an expert in compression technologies, PrismML focuses on shrinking the size of LLMs without significantly sacrificing performance. Bonsai 2 27B maintains 98% of Qwen3.8’s benchmark scores, improving on the first Bonsai model released in March, which matched 95% of the original’s performance. The earlier Bonsai model has been downloaded over 11 million times, with smaller variants downloaded an additional 2.6 million times.

The startup’s compression technique involves converting model weights—normally stored as 16-bit values—into ternary weights represented by just three possible values: +1, −1, or 0. This approach drastically reduces the storage requirements while preserving the model’s reasoning capabilities.

PrismML is backed by investors including Khosla Ventures, Cerberus Capital, and Caltech, and counts Databricks co-founder Ion Stoica as an adviser. Stoica highlighted the significance of this technology, noting that it enables advanced AI models to run locally on devices, enhancing privacy and eliminating reliance on cloud computing.

Looking ahead, PrismML plans to apply its compression methods to even larger models with several hundred billion parameters. Hassibi expects that larger models will be easier to compress without losing intelligence, potentially improving performance parity further.

This development could reshape how AI is deployed by making powerful language models accessible on everyday devices, reducing latency, increasing privacy, and lowering costs associated with cloud-based AI services.