A new neural network model developed by Microsoft’s General Artificial Intelligence group uses just three distinct weight values: -1, 0, or 1, reducing overall complexity and computational resources. The “ternary” architecture achieves performance comparable to leading open-weight models across a range of tasks. This innovation sets it apart from previous research on quantization techniques that aim to simplify model weights.
Researchers drew inspiration from previous work on quantization and developed the first native, open-source 1-bit language model trained at scale. The new model is based on a 2 billion token training dataset of 4 trillion tokens. In contrast to post-training quantization methods, which can lead to significant performance degradation, this approach allows for true native training, enhancing capabilities.
The ternary system offers substantial advantages in computational efficiency and reduces memory footprints to hundreds of gigabytes, making it suitable for desktop CPUs. Despite the reduced precision, the model achieves comparable performance to leading full-precision models, opening up new possibilities for AI applications.
Source: https://arstechnica.com/ai/2025/04/microsoft-researchers-create-super%E2%80%91efficient-ai-that-uses-up-to-96-less-energy