Microsoft researchers have created a lightweight 1-bit large language model (LLM) called BitNet b1.58 2B4T, which is small enough to run efficiently on some CPUs, including the Apple M2 chip. This innovative model uses 1-bit weights with only three possible values: -1, 0, and +1, reducing memory consumption by a significant amount compared to mainstream AI models.
The team trained BitNet b1.58 2B4T on over 33 million books, which is more data than many leading LLMs have seen. The model scored relatively well against these models in most tests, outperforming some of them in benchmarks while using significantly less memory. In fact, it consumed only 400MB in non-embedded memory, which is less than 30% of what the next smallest model used.
This achievement has significant implications for energy efficiency and accessibility to AI technology. Lightweight LLMs like BitNet b1.58 2B4T can help run AI models locally on less powerful hardware, reducing dependence on massive data centers and making AI more accessible to people without access to high-end processors or GPUs.
The framework bitnet.cpp that supports this model is available on GitHub, allowing anyone to experiment with it. This development has the potential to democratize AI technology and make it more inclusive for a wider range of users.
Source: https://www.tomshardware.com/tech-industry/artificial-intelligence/microsoft-researchers-build-1-bit-ai-llm-with-2b-parameters-model-small-enough-to-run-on-some-cpus