Chris Olah

Hi! I lead interpretability research at Anthropic. I also used to do a lot of basic ML pedagogy (https://colah.github.io/). I think this post and its children have some important questions about modern deep learning and how it relates to our present research, and wanted to take the opportunity to try and clarify a few things.

When people talk about models "just predicting the next word", this is a popularization of the fact that modern LLMs are "autoregressive" models. This actually has two components: an architectural component (the model generates words one at a time), and a loss component (it maximizes probability).

As the parent says, modern LLMs are finetuned with a different loss function after pretraining. This means that in some strict sense they're no longer autoregressive models – but they do still generate text one word at a time. I think this really is the heart of the "just predicting the next word" critique.

This brings us to a debate which goes back many, many years: what does it mean to predict the next word? Many researchers, including myself, have believed that if you want to predict the next word really well, you need to do a lot more. (And with this paper, we're able to see this mechanistically!)

Here's an example, which we didn't put in the paper: How does Claude answer "What do you call someone who studies the stars?" with "An astronomer"? In order to predict "An" instead of “A”, you need to know that you're going to say something that starts with a…

Yes for sure. I am very math-averse (almost failed calculus 1, skated by pre-calc in high school... not my intelligence area) and though I don't have a full time job doing something in AI, I do feel like I can build models that solve real problems (and am doing so in an internship right now).

Probably start with Andrew Ng's Machine Learning course. It has a significant amount of Math in it-- try to understand it, but seriously do not worry about it. Just get the high level concepts, try to get some intuition on machine learning ideas and techniques. You don't need to do the assignments or work too hard on the course (but obviously it's helpful if you do).

Then read these. Don't worry if you're still confused at first. It's fine. Just go through 'em kinda slowly and try to see what's going on.
https://iamtrask.github.io/2015/07/12/basic-python-network/
http://karpathy.github.io/neuralnets/

By now if you're still into it, I highly recommend Chris Olah's blog: http://colah.github.io/
It has some pretty complicated ideas in there, but the articles are illustrated and explained very well so you can get more of a feel for Neural Networks while also getting very excited about them.

Then it's probably time to start really building things. I would use Keras (https://keras.io/) at first, because it's very easy to get cool results without understanding everything under the hood. Do a tutorial or two, it's pretty intuitive and if you've done everything above you should understand…

Although probably not sufficient for AGI, network architecture is essentially guaranteed to be important, because of both ample empirical evidence of the importance of architectures and ample reason, from facts about numerics, to believe that it is important.

In the first category (empirical evidence),

- The discrete leap from non-LSTM RNN to LSTM network performance on NLP was essentially due to a "better factoring of the problem": breaking out the primitive operations that equate to an RNN having "memory" had a substantial effect on how well it "remembered."

- The leap in NMT from LSTM seq2seq to attention-based methods (the Transformer by Google) is another example. Long-distance correlations made yet another leap because they are simply modeled more directly by the architecture than in the LSTM.

- The relation network by DeepMind is another excellent example of a drop-in, "pure" architectural intuition-motivated replacement that increased accuracy from the 66% range to the 90% range on various tasks. Again, this was through directly modeling and weight-tying relation vectors through the architecture of the network.

- The capsule network for image recognition is yet another example. By shifting the focus of the architecture from arbitrarily guaranteeing only positional invariance to guaranteeing other sorts, the network was able to do much better at overlapping MNIST. Again, a better factoring of the problem.

These developments all illustrate that picking the…

No, you don't need the degree.

Denny Britz, He drops school (his phd) to go into industries.

Christopher Olah: Same story, at least he doesn’t communicate about his education (http://colah.github.io/cv.pdf).

Both are well-known for their expertise, and followed by a large amount of people in the deep learning community.

You can def become an great machine learning engineer who used deep learning technique, without going to any university.

---
Ian Goodfellow:

"One of my biggest heroes is Geoffrey Hinton. His PhD was in experimental psychology ( . Biographical Sketch ).

In mechanical engineering, I think you learn quite a lot of the mathematical tools that are used in machine learning, so you won’t be starting from scratch.

In general, we often find that revolutions in one field happen when someone from another field arrives with fresh ideas that developed elsewhere."

(https://www.quora.com/Would-you-encourage-people-from-another-field-mechanical-engineering-to-learn-ML)

How can someone with almost no technical knowledge learn ML?

Ian Goodfellow::"It’s important to master some of the basics, like linear algebra, probability, and python programming.

But you don’t need to know all of linear algebra (I don’t think I’ve ever used a QR decomposition outside the classroom), all of probability (we don’t usually do much involving combinatorics / re-ordering of sequences in deep learning), or all of python programming (a lot of the more obscure language features are…

I wish I knew what it was like to have the math background, and don't at all mean to suggest one should not bother, but as a person on a team of people who have made a few production NN models over the past year: it is completely possible to conceptually and practically grasp iterated reverse differentiation with convolutions well enough to use deep learning to do novel work, without having the deep background math knowledge. For example, I barely know what a Support Vector Machine is, nor could I do a linear regression without a lot of hand holding. But I can design a passable tensorflow model and improve it.

It would definitely be very hard to do any meaningful research from this position, but I know enough to be useful (er, or dangerous) and can read papers and code to keep up with recent advancements, and try things out like different convolution designs, layer designs, gates, functions, feedback, etc.

(On the topic of reading: Karpathy[1] and Colah[2]'s posts have a wealth of introductory conceptual information and images in them, and helped our team discussions a lot while we've learned.

1 - https://karpathy.github.io/

2 - https://colah.github.io/)

If we succeed in training an accurate enough machine learning algorithm, I think the primary purpose will be to screen for asymptomatic atrial fibrillation, and then use a Holter or Zio Patch for a final diagnosis.

The main benefit is that you can prevent cryptogenic strokes, which are often caused by undiagnosed atrial fibrillation:
http://www.nejm.org/doi/full/10.1056/NEJMoa1311376

For machine learning, I think you're on exactly the right path. The Udacity TensorFlow course taught by the Google Brain team is good; if you can complete that, you'll be as good as anyone at applying neural networks. The Stanford courses (http://cs231n.stanford.edu/, http://cs224d.stanford.edu/) are likewise great, as is Chris Olah's blog for intuition (http://colah.github.io/).

And down the road, if you're ever looking for an internship, definitely let us know! brandon@cardiogr.am.