Mistral Large 2 is the latest version of Mistral AI’s flagship language model. It has a significant improvement in code generation, mathematics, and multilingual capabilities compared to its previous versions. The new model features 123 billion parameters, a context window of 128,000 tokens, and aims to challenge industry leaders in performance and efficiency.
Mistral Large 2 performs well across various benchmarks. On code generation tasks like HumanEval and MultiPL-E, it outperforms Llama 3.1 405B and scores just below GPT-4. In mathematics, particularly on the MATH benchmark (zero-shot), Mistral Large 2 ranks second only to GPT-4.
The model’s multilingual capabilities have also seen a substantial boost. On the Multilingual MMLU benchmark, Mistral Large 2 surpasses Llama 3.1 70B base by an average of 6.3% across nine languages and performs on par with Llama 3 405B.
Despite its large size, Mistral AI designed the model for single-node inference, emphasizing throughput for long-context applications. The company is making Mistral Large 2 available on its platform, la Plateforme, and has released the weights for the instruct model on HuggingFace for research purposes.
The CEO of Mistral AI stated that Mistral Large 2 sets a new frontier in terms of performance-to-cost ratio on evaluation metrics. The pretrained version achieves an 84.0% accuracy on MMLU, establishing a new point on the performance/cost Pareto front for open models.
Mistral Large 2 has undergone extensive training on source code and has shown comparable performance to leading models like GPT-4, Claude 3 Opus, and Llama 3 405B in coding tasks. The company also focused on enhancing the model’s reasoning capabilities and reducing hallucinations, resulting in improved performance on mathematical benchmarks.
Mistral Large 2 excels in instruction-following and conversational tasks, with particular improvements in handling precise instructions and long, multi-turn conversations. Its release signals intensifying competition in the AI language model space, positioning it as a formidable option for both research and potential commercial applications.
As AI models continue to grow in size and capability, Mistral AI’s focus on efficiency and single-node inference highlights an important trend in balancing performance with practical deployment considerations.
Source: https://www.maginative.com/article/mistral-ai-unveils-mistral-large-2-a-powerful-new-language-model/