Apple has published a research paper that suggests AI reasoning models have clear limits when it comes to solving complex problems. The company’s findings undermine developer arguments that these models are useful for tasks traditionally solved by humans. Researchers used complex logic puzzles, such as the Tower of Hanoi, and found that even with full “budget” for thinking, models stopped spending tokens on reasoning through the problem further beyond a certain level of complexity.
This research throws into doubt claims from other companies, including Google and Anthropic, about the capabilities of their reasoning models. The paper’s findings suggest that these models are more likely to fail when allowed to work on a problem for too long, rather than improving with increased complexity. As a result, developers may need to implement new approaches and architectures to solve this issue, which could impact customer trust in “thinking” models.
The limitations of AI reasoning models highlight the challenges of developing technology that can truly think and reason like humans. The findings also raise questions about the future of job automation and the potential impact on workers.
Source: https://www.itpro.com/technology/artificial-intelligence/apple-ai-reasoning-research-paper-openai-google-anthropic