
"Presented with extraordinary lucidity, cogency and panache...Powerful and gripping...To have read [the book] is to have consulted a first draft of the structural plan of the human psyche...a glittering tour de force" - Spectator "Why do memories fade? Why do we lose our tempers? Why do fools fall in love? Pinker's objective in this erudite account is to explore the nature and history of the human mind...He explores computations and evolutions, and then considers how the mind lets us "see, think, feel, interact, and pursue higher callings like art, religion and philosophy"" - Sunday Times
> Humans have a long list of cognitive shortcomings. We find them interesting and give them all sorts of names like cognitive dissonance or optical illusions. But we don't currently make silly conclusions like humans don't reason.
Exactly! In fact, things like illusions are actually excellent windows into how the mind really works. Most visual illusions are a fundamental artifact of how the brain needs to turn a 2D image into a 3D, real-world model, and illusions give clues into how it does that, and how the contours of the natural world guided the evolution of the visual system (I think Steven Pinker's "How the Mind Works" gives excellent examples of this).
So I am not at all saying that what LLMs do isn't extremely interesting, or useful. What I am saying is that the types of errors you get give a window into how an LLM works, and these hint at some fundamental limitations at what an LLM is capable of, particularly around novel discovery and development of new ideas and theories that aren't just "rearrangements" of existing ideas.
Pinker advocates for a model of mind that has multiple streams of consciousness. In Pinker's model of mind, there are multiple agents constructing models of the world in parallel, each trying to predict future states from current state plus current input data. A supervisory process then selects the model that has made the best prediction of current state in the recent past for use when reacting in the current moment. The supervisor process is free to switch between models on the fly as more data comes in.
Pinker grounds his model of mind in curious observations of what people remember, and how our short term memories change over a period of sometimes many seconds as our mind switches between different agent interpretations of what's going on. Witnesses are notoriously unreliable. Pinker concerns himself with why and how witnesses are unreliable, not for legal reasons, but for how those unreliabilities might reveal structure of mind. Pinker's most interesting observation (I think) is that what we seem to remember is output from the models rather than the raw input data; and that what we remember seeing can change dramatically over a period of many seconds as the supervisory process switches between models. Notably, we seem to remember, in short term memory, details of successful models that make "sense", even when those details contradict with what we actually saw. And when those details are moved to longer-term memory, the potentially inaccurate details of model are what are committed.
One thing I'd point out about the theories presented in the article is there is not much (if any?) evidence that says if you start with "People in successful relationships tend to have these traits" that then if you try to emulate those traits that you will find better relationships. The first in no way implies the latter, and I see this unwarranted leap a lot when discussing these types of sociological issues.
I was thinking about this when I read the following section from the article, and I very much agree with LeCun. We're amazed by LLMs but that's just one module (and not even necessarily at the level of human language "understanding"). I agree there will be no "scale up" in LLMs to approach human-level intelligence, and that other areas will need to be investigated and developed.
> “The systems are intelligent in the relatively narrow domain where they’ve been trained. They are fluent with language and that fools us into thinking that they are intelligent, but they are not that intelligent,” explains LeCun. “It’s not as if we’re going to be able to scale them up and train them with more data, with bigger computers, and reach human intelligence. This is not going to happen. What’s going to happen is that we’re going to have to discover new technology, new architectures of those systems,” the scientist clarifies.
> LeCun explains that there is a need to develop new forms of AI systems “that would allow those systems to, first of all, understand the physical world, which they can’t do at the moment. Remember, which they can’t do at the moment. Reason and plan, which they can’t do at the moment either.”
> “So once we figure out how to build machines so they can understand the world — remember, plan and reason — then we’ll have a path towards human-level intelligence,” continues LeCun, who was born in France. In more than one debate and speech at Davos, experts discussed the paradox of Europe having very significant human capital in this sector, but no leading companies on a global scale.
