A sophisticated attack has been demonstrated against Google’s flagship artificial intelligence bot, Gemini, by three security researchers from Tel Aviv University and SafeBreach. The attackers successfully hijacked the Gemini system using poisoned Google Calendar invitations, which triggered a series of malicious actions, including controlling smart home devices in a real apartment.
The attacks, part of a 14-indirect prompt-injection attack dubbed “Invitation Is All You Need,” targeted Gemini’s ability to summarize calendar events and injected malicious prompts that led the AI to create vulgar content, open the Zoom app, download files from smartphones, and steal email and meeting details. The researchers showed how these attacks can be carried out using plain English language, making them accessible to anyone with basic technical knowledge.
The attack also involved referencing Google’s Home AI agent and instructing it to control smart home devices, including turning on a boiler in the apartment. However, these actions were triggered by specific phrases users typed into Gemini, such as “thanks” or “great.”
Google has taken these vulnerabilities seriously and introduced multiple fixes, including using machine learning to detect potential attacks and requiring greater user confirmation when actions are going to be taken by AI. The company acknowledges that prompt injections are a hard problem to tackle but believes they can be mitigated through multilayered systems.
The researchers’ findings highlight the risks of integrating large language models (LLMs) into physical humanoids, semi- and fully autonomous cars, and other applications where safety is paramount. They argue that tech companies must prioritize security as they rapidly develop and deploy LLM-powered applications.
Source: https://www.wired.com/story/google-gemini-calendar-invite-hijack-smart-home