A BBC study has found that more than half of the AI-generated news summaries from popular chatbots struggled with factual inaccuracies and distortions. The research, which examined four major AI models, discovered that 51% of responses contained “significant issues,” while 19% included incorrect statements and dates.
The study tested the accuracy of OpenAI’s ChatGPT, Google Gemini, Microsoft Copilot, and Perplexity by asking them to summarize 100 BBC news articles. Journalists then reviewed their answers, highlighting major concerns. Examples of inaccuracies included a claim that the UK’s National Health Service advises against vaping for quitting smoking, when in fact it recommends vaping as an alternative.
Gemini was found to have raised the most concerns, with 46% of its responses flagged for accuracy issues. The BBC has previously addressed similar concerns with Apple’s AI-powered news summaries and is now calling on tech companies to address these issues. Deborah Turness, CEO of BBC News and Current Affairs, urged tech companies to work together with the news industry and government to prevent AI-distorted headlines from causing harm.
The study’s findings highlight the need for more robust quality control measures in AI-generated content. As the use of AI in journalism continues to grow, it is essential that these models are developed and tested with a focus on accuracy and trustworthiness.
Source: https://www.theverge.com/news/610006/ai-chatbots-distorting-news-bbc-study