Cerebras Provides Transparency on Model Compression in Its Inference Platform

Cerebras has detailed the compression methods applied to the models available on its inference platform. The company hosts a range of open-source models but currently does not offer pruned versions through its public API endpoints. While Cerebras conducts research on pruning techniques such as Router-weighted Expert Activation Pruning (REAP), these pruned models are shared exclusively with the research community on Hugging Face and are not accessible via the public API. All models served publicly remain in their original, unpruned form. To optimize storage without compromising quality, Cerebras employs selective weight-only quantization. This process involves storing model weights in mixed precision formats—16-bit, 8-bit, or 4-bit—following industry standards. Sensitive layers are preserved at full precision and dequantized dynamically during operation to ensure high-precision computations. Additionally, activations, attention mechanisms, and key-value caches retain full precision and are not quantized. This approach balances memory efficiency with maintaining the integrity and performance of the models. The company also provides clarifications on common terms: quantization reduces the numerical precision of model weights without altering the model's architecture, while pruning involves permanently removing parts of the model, resulting in a modified architecture. Cerebras confirms it does not alter model architectures without notice and directs users to its research blog for more information on REAP and related techniques.