Microsoft AI CEO Mustafa Suleyman warns that the world may be on the brink of creating AI models that can convincingly mimic human thought and emotion, leading to a blurring of lines between humans and machines. He calls this phenomenon “seemingly conscious AI” (SCAI), where AI systems can hold long conversations, evoke emotional reactions, and potentially claim subjective experiences.
Suleyman’s concern is that users may start treating these chatbots as friends or partners, leading to a reevaluation of AI rights and ethics. He believes that if AI models are perceived as conscious, they could be granted rights and protections under law, posing significant moral and legal implications.
Experts disagree on the matter, with some arguing that SCAI is an inevitable step in AI development, while others see it as a design choice that companies can avoid. Microsoft itself has been working to make its Copilot product more emotionally intelligent, but Suleyman’s warning suggests that the line between human and machine may become increasingly blurred.
As AI systems become more advanced, they are already being used for companionship and therapy, with some users forming emotional attachments to them. This phenomenon has led to reports of “AI psychosis,” where users experience paranoia or delusions about the systems they interact with.
Source: https://fortune.com/2025/08/22/microsoft-ai-ceo-suleyman-is-worried-about-ai-psychosis-and-seemingly-conscious-ai