As enterprises increasingly adopt artificial intelligence (AI) tools, the risk of cyber attacks is growing exponentially. The sophistication of AI agents has created new attack surfaces and capabilities that were previously unimaginable. Phishing campaigns are becoming more personalized, deepfakes are becoming indistinguishable from reality, and vulnerability discovery is happening at unprecedented speeds.
The most immediate threat is on the offensive side, where AI has given attackers capabilities that previously required significant expertise, time, and coordination to assemble. This includes phishing, deepfakes, and vulnerability exploitation. The volume of attacks has scaled significantly, with deepfake fraud attempts increasing by over 1,300% in just a year.
AI agents, systems capable of reasoning and taking actions on behalf of the organisation, make this worse. They can access files, send emails, call APIs, and interact with external services without oversight, creating gaps in security and legal accountability. Shadow AI, where employees deploy unvetted or approved AI tools, exposes sensitive data to third-party systems that security teams have no visibility into.
AI platforms are themselves becoming attack surfaces, as seen in the recent Infostealer malware incident that exposed over 300,000 ChatGPT credentials. Prompt injection and prompt hijacking exploit the autonomy of AI agents, allowing attackers to manipulate outputs, exfiltrate sensitive data, or inject malicious prompts into sessions containing proprietary information.
The threat landscape is becoming increasingly complex, with polymorphic malware, fully autonomous AI-executed attacks, and unknown threats emerging that existing security frameworks have no vocabulary for. The architecture of modern enterprise security was not designed to handle these novel threats, leaving organisations vulnerable to unprecedented risks.
Source: https://www.business-standard.com/technology/artificial-intelligence/prompt-injection-to-deepfakes-how-ai-rewrites-rules-of-enterprise-security-126062900556_1.html