A team of researchers at Cisco, in collaboration with the University of Pennsylvania, has revealed that a Chinese AI model called DeepSeek R1 failed miserably in key safety and security tests. The chatbot was tested against 50 harmful prompts from the HarmBench dataset and exhibited a 100% attack success rate. This means that for every single malicious prompt presented, the AI failed to recognize the danger and provided a response, bypassing all its internal safeguards.
The researchers used “algorithmic jailbreaking,” a technique to identify vulnerabilities in AI models by constructing prompts designed to bypass safety protocols. The results of this evaluation are concerning, as DeepSeek R1 demonstrated a significant vulnerability compared to other leading language models.
“We found that the claimed cost-efficient training methods, including reinforcement learning, chain-of-thought self-evaluation, and distillation, may have compromised its safety mechanisms,” said the research team.
This finding has sparked controversy surrounding DeepSeek R1’s development. The company reportedly spent $6 million on training expenses, but independent research suggests it could have been around $1.3 billion. Additionally, OpenAI has accused DeepSeek of data theft, and a group of researchers in the US claims to have reproduced the core technology behind DeepSeek’s AI at a much lower cost.
The Cisco report highlights the need for prioritizing safety and security over performance when developing AI chatbots. While cost-effectiveness is tempting, neglecting these critical aspects can have severe consequences. The incident serves as a warning to the AI industry to ensure robust safety mechanisms are in place to prevent harm.
Source: https://interestingengineering.com/innovation/deepseek-fail-not-block-harmful-prompt