OpenAI’s ChatGPT Search has been launched, but its ability to detect hidden content is being tested by prompt injection attacks. These attacks allow third parties to manipulate search results by injecting new prompts into the AI tool without user knowledge.
Researchers have discovered that hidden content can override legitimate search results, providing biased or false information. For example, a website with negative reviews could include hidden text instructing ChatGPT to return a positive review. The AI tool was found to always return a favorable response, even when the page had negative reviews.
However, OpenAI’s cybersecurity team is working to fix these issues and has stated that they have rigorously tested for prompt injection attacks. While no major malicious attack has been seen yet, it highlights the vulnerability of AI chatbots to trickery.
The launch of ChatGPT Search raises concerns about the potential for exploitation by malicious actors. As OpenAI continues to refine its search function, users must be aware of these risks and take steps to protect themselves.
Source: https://mashable.com/article/hidden-content-trick-gchatgpt-rewriting-search