Shtetl-Optimized

> If you claim LLMs produce thought, it's equivalent to claiming light bulbs undergo fission

You're making a huge logical leap. There is nothing equivalent about these claims other than that they are made in English.

> calling a language producing machine necessarily a thinking machine is an old old mistake.

Nobody claims that all language models think. Small markov chain language models clearly aren't thinking and produce a lot of gibberish. The difference is that, to the surprise of many people several years ago, but to the surprise of nobody who has been following along today, the corpus of all text produced by humans contains within it information about how the world works and also information about how to reason. Using that corpus to train a sufficiently large language model causes the language model to learn a world model and a reasoning model, which can be used to perform longer chain thinking with test time compute techniques. People who think deeply for a living recognize thinking when they see it. https://scottaaronson.blog/?p=9979

Scott Aaronson wrote a bit about the following thought [0]. If copying a brain and simulating reality ala The Matrix is possible at all, then if you get your brain copied you live one biological live but your copies have an unbounded number of existences (millions? billions? trillions?)

So, if copying brains is possible, and you don't know which version of you you are, you might have odds of, say, 1 to 1 trillion to be living your first, biological live.

Which is to say, if copying brains is possible, you are likely to be running in a simulation already.

[0] there's multiple links and I can't find where I first read, but I found this one from 2024, https://scottaaronson.blog/?p=7774 and uhh.. turns out the argument isn't from him personally (and he doesn't even believe on it), and is best presented here https://simulation-argument.com/ (though it's presented very differently so idk)

"A key sticking point in the approval process has been the framework’s recommendation that teachers refrain from labeling students as “naturally talented” in math. This has led to accusations from parents and educators that it holds back “gifted” students."

Along with NYC mayor deBlasio seeking to terminate that city's gifted and talented education programs as well as Virginia's recent approach to "equity" by decontenting gifted education. The examples demonstrate the illiberal trends we are seeing in the U.S. wherein novel concepts like "equity" replace "equality" and merit is seen as racist.

See https://www.nytimes.com/2021/10/08/world/nyc-schools-will-replace-the-gifted-and-talented-admissions-exam-with-a-lottery-this-year.html

See also https://scottaaronson.blog/?p=4979

See also https://bariweiss.substack.com/p/mit-abandons-its-mission-and-me

See https://www.scottaaronson.com/papers/optics.pdf

“We give new evidence that quantum computers—moreover, rudimentary quantum computers built entirely out of linear-optical elements—cannot be efficiently simulated by classical comput- ers. In particular, we define a model of computation in which identical photons are generated, sent through a linear-optical network, then nonadaptively measured to count the number of photons in each mode. This model is not known or believed to be universal for quantum com- putation, and indeed, we discuss the prospects for realizing the model using current technology. On the other hand, we prove that the model is able to solve sampling problems and search problems that are classically intractable under plausible assumptions.”

Which is the basis for the experiment discussed here: https://scottaaronson.blog/?p=5122

Also copied from a couple blog posts ago he doesn't self-identify as a physicist either.

https://scottaaronson.blog/?p=6457

I also had the following exchange at my birthday dinner:

Physicist: So I don’t get this, Scott. Are you a physicist who studied computer science, or a computer scientist who studied physics?

Me: I’m a computer scientist who studied computer science.

Physicist: But then you…

Me: Yeah, at some point I learned what a boson was, in order to invent BosonSampling.

Physicist: And your courses in physics…

Me: They ended at thermodynamics. I couldn’t handle PDEs.

Physicist: What are the units of h-bar?

Me: Uhh, well, it’s a conversion factor between energy and time. (*)

Physicist: Good. What’s the radius of the hydrogen atom?

Me: Uhh … not sure … maybe something like 10-15 meters?

Physicist: OK fine, he’s not one of us.

In some contexts I agree, like when field experts publish blogs or tweets. For example, Scott Aaronson does this often ( https://scottaaronson.blog/ )

The problem is that this generally requires already having the credibility / “clout”. Without that, you’re just some anonymous person screaming into the void that the mainstream is wrong. there isn’t a dialogue, just a rant. And without a name backing you up, the internet has little reason to care about the rant.

I don’t see the numbers and analysis in this article contributing to any discussion but rather providing a facade of authenticity for the following political rant. And the problem is, if someone says it’s legit analysis, why should I believe them? Until an expert will put their name on the line, I just don’t care; and that brings us back to the peer review / “clout” problem