Gary Marcus

> Now there is no sign of actual AGI happening any time soon

Do you genuinely hold this position, or do you not realize how far the goalposts have shifted?

In 2022, prominent AI critic Gary Marcus offered to bet $100,000 that we wouldn't have AGI by 2029. https://garymarcus.substack.com/p/dear-elon-musk-here-are-five-things Because the definition of AGI is unclear, he defined that AGI would be achieved if an AI model could do THREE of the five following tasks:

- In 2029, AI will not be able to watch a movie and tell you accurately what is going on (what I called the comprehension challenge in The New Yorker, in 2014). Who are the characters? What are their conflicts and motivations? etc.

- In 2029, AI will not be able to read a novel and reliably answer questions about plot, character, conflicts, motivations, etc. Key will be going beyond the literal text, as Davis and I explain in Rebooting AI.

- In 2029, AI will not be able to work as a competent cook in an arbitrary kitchen (extending Steve Wozniak’s cup of coffee benchmark).

- In 2029, AI will not be able to reliably construct bug-free code of more than 10,000 lines from natural language specification or by interactions with a non-expert user. [Gluing together code from existing libraries doesn’t count.]

- In 2029, AI will not be able to take arbitrary proofs from the mathematical literature written in natural language and convert them into a symbolic form suitable for symbolic verification.

Today's AI models can…

Hey, I'm definitely on your side of the Great AI Wars--and definitely share your thoughts on the overall framing--but I think you're missing the serious nature of this contribution:

1. Small correction, it's actually a whole book AFAIK, and potentially someday soon, a class! So there's a lot more thought put in then the typical hot-take blog post. I also pop into one of these guy's replies on Bluesky to disagree on stuff fairly regularly, and can vouch for his good faith, humble effort to get it right (not something to be taken for granted!)

2. RE:“the AI has no ground truth”, I'd say this is true, no matter how often they're empirically correct. Epistemological discussions (aka "how do humans think") invariably end up at an idea called Foundationalism, which is exactly what it sounds like: that all of our beliefs can be traced back to one or more "foundational" beliefs that we either do not question at all (axioms) or very rarely do (premises on steroids?). In that sense, this phrase is simply recalling the hallucination debates we're all familiar with in slightly more specific, long-standing terms; LLMs do not have a systematic/efficient way of segmenting off such fundamental beliefs and dealing with them deliberately. Which brings me to...

3. RE:“can’t reason logically”, again this is a common debate that I think is being specified more than usual here. A lot of philosophy draws a distinction between automatic and deliberate cognition. I give credit to Kant for the…

You’re still failing to understand that a model being able to output a prediction of something is not the same thing as it “knowing” that thing. The Newton-Raphson method doesn’t “know” what the root of a function is, it just outputs an approximation of it.

> It’s extremely silly then don’t you think to make such bold declarations on what doesn’t have it?

I don’t find it particularly bold to respond to your assertion that a piece of mathematics is sentient life by stating that you haven’t proven that it is, and that in the absence of that proof, the most rational position is to continue to believe that it is not, as we have done for millennia. The burden of proof is on you.

> if it does anything a conscious being does

You haven’t shown that it can do anything that only conscious beings can do.

Being able to generate a passable approximation of text that might follow some prompt doesn’t mean that it understands the prompt, or its answer. As an obvious example, if you give LLMs maths problems, they change their answers if you change the names of the people in the question. They’re not actually doing maths.

> Notice anything? It’s not just that the performance on MathGLM steadily declines as the problems gets bigger, with the discrepancy between it and a calculator steadily increasing, it’s that the LLM based system is generalizing by similarity, doing better on cases that are in or near the training set, never, ever getting to a complete, abstract, reliable…

These tools have already peaked usage, and even its greatest proponents are questioning its viability, see:

https://garymarcus.substack.com/p/is-vibe-coding-dying

I'm under the impression a lot of these tools are:

1. Aggressively pushed by VCs on company boards.

2. Productive of code that is not maintainable and becomes very difficult to deal with when it is non-trivial. 3. Not what customers want.

Don't get me wrong, I use LLMs and LLM tech in my work*, they are useful and interesting products, but they are a small part of the work and a small part of the product. Sure, there are people who use them extremely effectively, but those people are offset by those who have LLMs write code they don't understand and do not review (leading to a scenario where code is effectively ghost code often without even provenance back to the LLM that wrote it).

It seems to me that layoffs in tech are partly a cultural contagion in executive circles, but more importantly it seems to me like offshoring is much more responsible.

[*]: Mainly for codemods and reorganization of code, where I'm not really changing the intent of the code but its structure.

https://garymarcus.substack.com/p/openais-amazing-but-vastly-oversold?r=8tdk6&utm_campaign=post-expanded-share&utm_medium=web&triedRedirect=true

https://garymarcus.substack.com/p/two-critical-updates-re-astra-and?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb4376bf6-f1ba-4471-88b5-4307e2581a40_1213x1027.png&open=false

As always, PR hype. Goalposts have not moved.

Guys, please use critical thinking. The haters don't hate by default, we hate because we're gaslit about this stuff every day and it's annoying. Extraordinary claims require proof, and they're not giving us information that would be essential to knowing if this is actually significant or not.

Actually, I think the industry is finally realizing that there is no intelligence there, especially with gpt3 which can figure out with great precision what statistically comes next, but there is zero understanding of the space-time conceptual meaning for gpt3 in that answer - its not designed to do anything but figure out statistically what is most likely to come next.

Gary Marcus has been doing a good job of exposing this: https://garymarcus.substack.com/

Another key piece of evidence, the failure of all FSD attempts trying to use ml/dl thinking its more than just statistics.