AI Model Creates False Policy, Causes Subscription Cancellations

A developer using the AI-powered code editor Cursor noticed a strange issue where switching between machines instantly logged them out, breaking common workflows for programmers who use multiple devices. When they contacted Cursor support, they were told it was expected behavior under a new policy, but no such policy existed and the support agent was actually a bot.

This incident marks one of several recent cases of AI “hallucinations” causing potential business damage. AI models often invent plausible-sounding but false information instead of admitting uncertainty. In this case, Cursor’s lack of human oversight led to a wave of complaints and cancellation threats on platforms like Hacker News and Reddit.

The incident began when a user noticed that switching between devices would terminate their sessions. They reported the issue to support, who replied with a response that sounded definitive and official. However, it was later revealed that Sam, the support agent, was not human.

As users took the initial post as confirmation of an actual policy change, several publicly announced their subscription cancellations on Reddit, citing the non-existent policy as their reason. This highlights the potential consequences of relying on AI systems in customer-facing roles without proper oversight.

Source: https://arstechnica.com/ai/2025/04/cursor-ai-support-bot-invents-fake-policy-and-triggers-user-uproar