LLMs can’t reason — they just crib reasoning-like steps from their training data
LLMs can’t reason — they just crib reasoning-like steps from their training data
Researchers at Apple have come out with a new paper showing that large language models can’t reason — they’re just pattern-matching machines. [arXiv, PDF] This shouldn’t be news to anyone here. We …
We suspect this research is likely part of why Apple pulled out of the recent OpenAI funding round at the last minute.
Perhaps the AI bros “think” by guessing the next word and hoping it’s convincing. They certainly argue like it.
🔥
Oh what a sweet, sweet tune to end a Sunday to
Arxiv paper link referenced in the article: https://arxiv.org/pdf/2410.05229
"sigh"
(Preface: I work in AI)
This isn't news. We've known this for many, many years. It's one of the reasons why many companies didn't bother using LLM's in the first place, that paired with the sheer amount of hallucinations you'll get that'll often utterly destroy a company's reputation (lol Google).
With that said, for commercial services that use LLM's, it's absolutely not true. The models won't reason, but many will have separate expert agents or API endpoints that it will be told to use to disambiguate or better understand what is being asked, what context is needed, etc.
It's kinda funny, because many AI bros rave about how LLM's are getting super powerful, when in reality the real improvements we're seeing is in smaller models that teach a LLM about things like Personas, where to seek expert opinion, what a user "might" mean if they misspell something or ask for something out of context, etc. The LLM's themselves are only slightly getting better, but the thing that preceded them is propping them up to make them better
IMO, LLM's are what they are, a good way to spit information out fast. They're an orchestration mechanism at best. When you think about them this way, every improvement we see tends to make a lot of sense. The article is kinda true, but not in the way they want it to be.
When you ask an LLM a reasoning question. You're not expecting it to think for you, you're expecting that it has crawled multiple people asking semantically the same question and getting semantically the same answer, from other people, that are now encoded in its vectors.
That's why you can ask it. because it encodes semantics.