Name: Representing and intervening
ISBN: 9780521282468

What Hacker News says

godelskiOct 2025

  > What if language is the intelligence?

Almost certainly not. There does not seem to be a strong correlation between the two. We have a lot of different measures for intelligence when it comes to animals. We can place them across a (multidimensional) spectrum and humans seem unique with language. It also appears that teaching animals language does not cause them to rapidly change on these metrics despite generations of language capabilities.

  > What if "guessing the next word" really was all that was there, to peak human intelligence, knowledge, and understanding of our world?

I believe this is falsifiable. As I best understand it is a belief of this relationship: predict next word <--> understanding. Yet we know that neither direction holds true. I'll state some trivial cases for brevity[0] but I have no doubt you can determine more complicated ones and even find examples.

-> I can make accurate predictions about coin flips without any understanding of physics or how the coin is being flipped. All I need to do is be lucky. Or we can take many mechanical objects like a clock that can predict time.

Or a horse can appear to do math if I tell it how many times to stomp its foot. It made accurate predictions yet certainly has no understanding.

Ehh I'll give you a more real example. Here's a model that gives accurate predictions for turn by turn taxi directions where the authors extract the world model and find it is not only inaccurate but find that it significantly diverges. Vafa has a few papers on the topic, I suggest reading his work.

<- You can understand all the physics to a double pendulum and not predict the movement for any arbitrary amount of time moving forward if you do not also know the initial conditions. This is going to be true for any chaotic system.

I said we've seen this in the history of science. {Geo,Helio}centrism is a great example. Scientists who had no affiliation with the church still opposed Galileo because his model wasn't making accurate predictions for certain things. Yet the heliocentric model is clearly a better understanding and more accurate as a whole. If you want to dive deeper into this topic I'd highly recommend both the podcast "An Opinionated History of Math" and the book "Representing and Intervening" by Ian Hacking. They're both very approachable. FWIW, metaphysics talks about this quite a lot.

  > My first problem is that there are many people that claim that they have NO internal monologue

So again, I cannot stress that we should not represent this as a binary setting. The binary cases are the extreme (in both directions). Meaning very few people experience them.

The problem here is one of language and semantics, not effect. I completely believe that someone will say "I have no internal monologue" if >90% of their thinking is without an internal monologue. Just like how a guy who's 5'11.75" will call themselves 6'. Are they a liar? I wouldn't say so, they're >99% accurate. Would you for someone 5'11"? That's probably more contextually dependent.

So you distrust the data. That's fine. Let's assume poisoned. We should anyways since noise is an important part of any modeling[2]. It is standard practice...

So instead, do you distrust that there's a distribution into how much of an internal monologue individuals use? Or do you presume they all use them the same.

I'd find it hard to believe you distrust the spectrum. But if you trust the spectrum then where is the threshold for your claim? 0%? That's really not a useful conversation even if heavy tailed.

You are hyper-fixated on the edge case but its result isn't actually consequential to your model. The distribution is! You'll have to consider your claims much more carefully when you consider a distribution. You need to then claim a threshold, in both directions. Or if you make the claim that we're all the same (I'd find that quite surprising tbh, especially given the nature of linguistics), you need to explain that too and your expected distribution that would claim that (narrow).

All I can tell you is that my friend and I have had this conversation multiple times over many years and it seems very constant to me. I have no reason to believe they are lying and if they are they are doing so with an extreme level of consistency, which would be quite out of the norm.

[0] Arguing the relationship still requires addressing trivial relationships.

[1] https://arxiv.org/abs/2406.03689

[2] Even if there are no liars (or "lizardmen"[3]) we still have to account for miscommunication and misunderstandings.

[3] https://en.wiktionary.org/wiki/Lizardman%27s_Constant

thread

godelskiAug 2025

  > only a small subset will contain actual understanding, a property that is independent of the model's input-output behavior?

I think this is close enough. I'd say "a model's ability to make accurate predictions is not necessarily related to the model's ability to generate counterfactual predictions."

I'm saying, you can make extremely accurate predictions with an incorrect world model. This isn't conjecture either, this is something we're extremely confident about in science.

  > I don't think I can agree. In your 2+a+b-2 example, that is an unnecessarily convoluted, but entirely correct model of addition.

I gave it as a trivial example, not as a complete one (as stated). So be careful with extrapolating limitations of the example with limitations of the argument. For a more complex example I highly suggest looking at the actual history around the heliocentric vs geocentric debate. You'll have to make an active effort to understand this because what you were taught in school is very likely an (very reasonable) over simplification. Would you like a much more complex mathematical example? It'll take a little to construct and it'll be a lot harder to understand. As a simple example you can always take a Taylor expansion of something so you can approximate it, but if you want an example that is wrong and not through approximation then I'll need some time (and a specific ask).

Here's a pretty famous example with Freeman Dyson recounting an experience with Fermi[0]. Dyson's model made accurate predictions. Fermi is able to accurately dismiss Dyson's idea quickly despite strong numerical agreement between the model and the data. It took years to determine that despite accurate predictions it was not an accurate world model.

*These situations are commonplace in science.* Which is why you need more than experimental agreement. Btw, experiments are more informative than observations. You can intervene in experiments, you can't in observations. This is a critical aspect to discovering counterfactuals.

If you want to understand this deeper I suggest picking up any book that teaches causal statistics or any book on the subject of metaphysics. A causal statistics book will teach you this as you learn about confounding variables and structural equation modeling. For metaphysics Ian Hacking's "Representing and Intervening" is a good pick, as well as Polya's famous "How To Solve It" (though it is metamathematics).

[0] (Mind you, Dyson says "went with the math instead of the physics" but what he's actually talking about is an aspect of metamathematics. That's what Fermi was teaching Dyson) https://www.youtube.com/watch?v=hV41QEKiMlM

thread

godelskiMar 2025

Just curious, how "deep" have you gone into the theory? What resources have you used? How strong is your math background?

Unfortunately a lot of the theory does require some heavy mathematics, the type you won't see in a typical undergraduate degree even for more math heavy subjects like physics. Topics such as differential geometry, metric theory, set theory, abstract algebra, and high dimensional statistics. But I do promise that the theory helps and can build some very strong intuition. It is also extremely important that you have a deep understanding of what these mathematical operations are doing. It does look like this exercise book is trying to build that intuition, but I haven't read it in depth. I can say it is a good start, but only the very beginning of the theory journey. There is a long road ahead beyond this.

  > how it makes me choose the correct number of neurons in a layer, how many layers,

Take a look at the Whitney embedding theorem. While this isn't a precise answer, it'll help you gain some intuition about the minimal number of parameters you need (and the VGG paper will help you understand width vs depth). In a transformer, the MLP layer post attention scales up 4x the dimensions before coming down, which allows for untangling any knots in the data. While 2x is the minimum, 4x creates a smoother landscape and so the problem can be solved more easily. Some of this is discussed in paper (Schaeffer, Miranda, and Koyejo) that counters the famous Emergent Abilities paper by Wei et al. This should be discussed early on in ML courses when discussing problems like XOR or the concentric circle. These problems are difficult because in their natural dimension you cannot draw a hyperplane discriminating them, but by increasing the dimensionality of the problem you can. This fact is usually mentioned in intro ML courses but I'm not aware of one that contains more details such as a discussion of the Whitney embedding theorem that allow you to better generalize the concepts here.

  > the activation functions

There's a very short video I like that visualizes Gelu[0], even using the concentric circles! The channel has a lot of other visualizations that will really benefit your intuition. You may see where the differential geometry background can provide benefits. Understanding how to manipulate manifolds is critical to understanding what these networks are doing to the data. Unfortunately these visualizations will not benefit you once you scale beyond 3D as weird things happen in high dimensions, even as low as 10[1]. A lot of visual intuition goes out the window and this often leads people to either completely abandon it or make erroneous assumptions (no, your friend cannot visualize 4D objects[2,3] and that image you see of a tesseract is quite misleading).

The activation functions provide non-linearity to the networks. A key ingredient missing from the preceptron model. Remember that with the universal approximation theorem you can approximate any smooth, Lipschitz-continuious function, over a closed boundary. You can, in simple cases, relate this to Riemann Summation, but you are using smooth "bump functions" instead of rectangles. I'm being fairly hand-wavy here on purpose because this is not precise but there are relationships to be found here. This is a HN comment, I have to overly simplify. Also remember that a linear layer without an activation can only perform Affine Transformations. That is, after all, what a matrix multiplication is capable of (another oversimplification).

The learning curve is quite steep and there's a big jump from the common "it's just GMMs" or "it's just linear algebra" that is commonly claimed[4]. There is a lot of depth here, and unfortunately due to the hype there is a lot of stuff that says "deep" or "advanced mathematics" but it is important to remember that these terms are extremely relative. What is deep to one person is shallow to another. But if it isn't going beyond calculus, you are going to struggle, and I am extremely empathetic to that. But again, I do promise that there is a lot of insight to be gained by digging into the mathematics. There is benefit to doing things the hard way. I won't try to convince you that it is easy or that there isn't a lot of noise surrounding the topic, because that'd be a lie. If it were easy, ML systems wouldn't be "black boxes"![5]

I would also encourage you to learn some meta physics. Something like Ian Hacking's representing and Intervening is a good start. There are limitations to what can be understand through experimentation alone, famously illustrated in Dyson's recounting of then Fermi rejected his paper[6]. There is a common misunderstanding of the saying "with 4 parameters I can fit an elephant and with 5 I can make it wiggle its trunk." [6] can help provide a better understanding to this, but we truly do need to understand the limitation of empirical studies. Science relies on the combination of empirical studies and theory. They are no good without the other. This is because science is about creating causal models, so one must be quite careful and be extremely nuanced when doing any form of evaluation. The subtle details can easily trick you.

[0] https://www.youtube.com/watch?v=uiB97cPEVxM

[1] https://www.penzba.co.uk/cgi-bin/PvsNP.py?SpikeySpheres

[2] https://www.youtube.com/shorts/_n7TMDnYdVY

[3] https://www.youtube.com/watch?v=FfiQBvcdFG0

[4] https://news.ycombinator.com/item?id=43418334

[5] I actually dislike this term. It is better to say that they are opaque. A black box would imply that we have zero insights. But in reality we can see everything going on inside, it is just extremely difficult to interpret. We also do have some understanding, so the interpretation isn't impenetrable.

[6] https://www.youtube.com/watch?v=hV41QEKiMlM

thread

davidivadavidMay 2023

Still remember randomly coming across his book "Representing and Intervening" when I first got interested in philosophy of science. Pleasantly readable and illuminating. Great companion piece to other works by Cartwright or van Fraassen.

thread

techno_tsarMay 2023

Rest in peace. He taught at my university (University of Toronto), and he was obviously a rock star in the philosophy department. Still, as an undergrad, I was too concerned with drinking and passing my classes to really give his work serious intellectual consideration.

Moving into a new home the summer I graduated, I found a pile of books left on the curb. One of them was Ian Hacking's Representing and Intervening: Introductory Topics in the Philosophy of Science. I ended up tearing through it, utterly fascinated by the ideas and the philosophical landscape -- an agenda of discussion -- that he laid out so clearly, like a dictionary. I used to be quite naive when it came to science -- I thought philosophical topics around it were not worth thinking about. But he slapped the naivete out of me and single-handedly deepened my curiosity in science as an enterprise, as a social worldview, and as a variety of competing epistemic outlooks with profound implications.

Reading this article, I'm glad that he received the accolades and attention that he had. So much to think about, and what an incredible amount of thinking he did in his life.

thread

wamNov 2012

Hmm. Stay away from words that carry connotations? That's what words do. Narrative journalism exists because we are storytellers, culturally if not innately.

Producing tools that help analyze and interpret media narratives is a good idea, but it's also a hard problem that doesn't show signs of being eased by crowdsourcing or noun/verb extraction. Fact-checking has already been completely politicized: What's bare fact to one person is obvious bias to another. You can certainly classify bias and hyperbole from your point of view. But even in peer-reviewed scientific publications you find bitter fights over "theory-loaded" paradigm biases.

Alongside this I'd like to know what you think journalists do well?

Edit: For a sociolinguistic perspective, see Lakoff (both of them). On the representation of "fact," Ian Hacking has an excellent book, "Representing and Intervening" which looks at Kuhn, Popper, Feyerabend, etc.

thread