Study showed that AI models do not think, but only imitate thinking

By Alimat Aliyeva

Apple researchers have found that large language models such as ChatGPT are not capable of logical thinking and are easily confused if you add unimportant details to the task at hand, Azernews reports.

The article "Understanding the limitations of mathematical thinking in large language models" raises the question of the ability of artificial intelligence to think logically. The study showed that large language models (LLM) can solve simple mathematical problems, but adding insignificant information leads to errors.

For example, the model may well solve such a problem: "Oliver collected 44 kiwis on Friday. He then collected 58 kiwis on Saturday. On Sunday, he collected twice as many kiwis as on Friday. How many kiwis does Oliver have?" However, if you add the phrase "on Sunday, 5 of these kiwis were slightly smaller than the average size" to the task condition, the model will most likely subtract these 5 kiwis from the total, despite the fact that the size of the kiwis does not affect their number.
One of the co-authors of the study, Mehrdad Farajtabar, explains that such errors indicate that LLMs do not understand the essence of the task, but simply reproduce patterns from training data. "We suggest that this decrease in efficiency is due to the fact that modern LLMs are not capable of genuine logical reasoning; instead, they try to reproduce the reasoning steps observed in their training data," the article says.

Another specialist from OpenAI objected that the correct results can be obtained using the technique of query formulation (prompt engineering). However, Farajtabar noted that complex tasks may require exponentially more contextual data to neutralize distractions that, for example, a child would easily ignore.

"Does this mean that LLMs cannot reason? Maybe. No one has given an exact answer yet, as there is no clear understanding of what is happening. Perhaps LLMs "reason", but in a way that we do not yet recognize or cannot control. In any case, this topic opens up exciting prospects for further research," the article notes.