Research indicates large language models (LLMs) face fundamental limitations, especially with compositional tasks requiring multistep reasoning, revealed through tests like Einstein's riddle. Despite being trained on vast datasets, LLMs struggle to apply learned patterns to solve complex problems, leading to erroneous outputs. Various studies highlight inherent mathematical and computational boundaries in transformer architecture, suggesting some tasks may remain beyond their capabilities. While improvements, such as enhanced training methods, are being explored, the limitations underscore the need for alternative AI approaches and deeper understanding of LLM operations.
https://www.quantamagazine.org/chatbot-software-begins-to-face-fundamental-limitations-20250131/