Yao Fu writes Notion-hosted research notes on large language models, covering emergent abilities, reasoning, instruction tuning, and inference optimization.
Commenters cite his piece tracing GPT's abilities to its sources, especially the idea that complex reasoning likely comes from training on code.
There are two camps, evident in this thread. one camp is 'its just a statistical model, it cant possibly know these things'
The other camp (that I'm in) sees that we might be onto something. We humans are obviously just more than a statistical model, but nonetheless learning words and how they fit together is a big part of who we are. With LLMs we have our first glimpse of 'emergent' behaviour from simple systems scaled massively. Whats are we if not a simple system scaled massively.
Check these links out:
Evidence that LLMs form internal models of what they learn about:
https://thegradient.pub/othello/
Evidence that training LLMs on code actually made them better at complex reasoning: https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1
John Carmack: https://dallasinnovates.com/exclusive-qa-john-carmacks-different-path-to-artificial-general-intelligence/ I think that, almost certainly, the tools that we’ve got from deep learning in this last decade—we’ll be able to ride those to artificial general intelligence.
A lot of the argument comes down to semantics about knowing and thinking. "An LLM can't think and a submarine cant swim"
For all models smaller than 62B, direct prompting outperforms CoT.
The first model where CoT outperforms direct prompting is Flan-cont-PaLM 62B on BBH. For 540B models, there are more settings where CoT outperforms direct prompting, but not all of them. Also, the number can be smaller than 540B. In Suzgun et. al. 2022, the authors show that the 175B InstructGPT and 175B Codex also have better CoT performance than direct prompting. Combining all the results, we get the two numbers 62B and 175B. So yes indeed, to enter the game of scale you do need a ticket to larger models than average.
However, there are also other large models like OPT, BLOOM, and the first version of GPT-3. They all have 175B, yet their CoT performance is significantly worse, or even cannot do CoT.
source: https://yaofu.notion.site/A-Closer-Look-at-Large-Language-Models-Emergent-Abilities-493876b55df5479d80686f68a1abd72f
Basically agree -- feels off-putting, but not technically a wrong detail to add. An additional reason it rubs me the wrong way, however, is that I believe open-source software code is especially critical to ChatGPT family's capabilities. Not just for code-related queries, but for everything! (e.g. see this "lineage-tracing" blog post: https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1#3e764cf66cd84fd481fb166ec43188ca)
Thus, I honestly think firms operating generative AI should be walking on eggshells to avoid placing blame on "open-source". Rather, they really should going out of their way to channel as much positive energy towards it as possible.
Still, agree the charitable interpretation is that this just purely descriptive.
Similar question, tho by my read, chain-of-thought seems to be standalone and the primary breakthrough. I asked the author[1].
I've worked in AI&ML since the 90s and both of these do seem much more intuitively promising than simply making larger nets. This seems borne out by the discussion in his previous post, that in Oct '21 OpenAI ran GPT-3 on it, solving 35% and they estimated:
"it appears likely that the 175 [billion link neural network] model would require at least two additional orders of magnitude of training data to reach an 80% solve rate.”
But instead 85% was achieved within a year using chain-of-thought, not a larger model.[2]
[1]https://twitter.com/PMayrgundter/status/1603224294124920832
[2]https://yaofu.notion.site/A-Closer-Look-at-Large-Language-Models-Emergent-Abilities-493876b55df5479d80686f68a1abd72f
ChatGPT cannot correctly perform step by step operations within strict formal systems. On the other hand, it can provide a high level sketch of how tomathematically prove something[1].
I think this is amazing, because this is the opposite of how we normally think about computers. ChatGPT is highly intuitive instead of a formal reasoning system. No doubt some future AIs will combine this intuition with checks against formal verification engines to check correctness.
[1] Section 6 of https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1
> There's a line of thought that if we're impressed with what we have, if it just gets bigger maybe eventually 'reasoning' will just emerge as a side effect. This is somewhat unclear and not really a strategy per se.
A recent analysis revealed that training on code might be the reason GPT-3 acquired multi-step reasoning abilities. It doesn't do that without code. So it looks like reasoning is emerging as a side effect of code.
(section 3, long article) https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1