Deep Learning

What Hacker News says

1. gmwhitebox_devApr 2025
   Thank you!

   To approach it from first principles, I didn’t really follow any one specific tutorial or course. I really just started from the bottom up. Starting with my Tensor module, I familiarized myself with the math operations, backprop, computation graphs, autodiff… etc. That was the hardest part honestly, but it set the foundation for everything else in the system.

   Once I had that working, the rest (activations, loss functions, optimizers, layers, transformers) started to make a lot more sense. Writing it all myself gave me full control, removed the abstraction, and helped me to really internalize how each part of the system fits together, and why it works - not just how.

   Here’s some resources I found helpful, I also have some links to additional resources in the project readme:

   - Deep Learning Foundations and concepts: Book by Christopher Bishop. Mainly covers theory and statistical ML.

   - Natural Language Processing with Transformers: Book by Lewis Tunstall, Leandro von Werra, and Thomas Wolf (Hugging Face). Good for understanding real world NLP/ LLMs.

   - UvA Deep Learning Tutorials: Website for building and understanding DL modules, has a lot of project-based notebooks (tutorial 7 on GNNs was very helpful).

   - Deep Learning: Book by Goodfellow, Bengio & Courville. Covers a lot of foundational theory and math.

   - Stanford’s CS231 Course: This is fully available online, with lecture videos and coding walk-throughs. Super helpful for learning about backprop, CNNs, and deep nets, etc.

   - The Annotated Transformer: Website by Harvard NLP. This goes over the ‘Attention is All You Need’ paper in an understandable way with example code

   - codingvidya.com: A good aggregator for finding ML books and learning resources.

   - Andrej Karpathy’s YouTube channel, and his micrograd and tinygrad Repos are super helpful resources, especially when learning by building from scratch.

   I’m still learning as I go, but I’m happy to share what’s worked for me so far! Hope this helps!

   Show morethread
2. larveAug 2024
   I actually find the quality of programming books to have starkly increased in the last decade. I find a lot of manning's and o'reilly's release to have a pretty long shelf-life.

   For example, I really enjoyed and often go back to:

   - https://www.oreilly.com/library/view/building-event-driven-m...

   - https://www.oreilly.com/library/view/designing-data-intensiv...

   - https://www.manning.com/books/100-go-mistakes-and-how-to-avo...

   - https://www.amazon.com/Systems-Performance-Brendan-Gregg/dp/...

   And more recently:

   - https://www.manning.com/books/build-a-large-language-model-f...

   - https://www.manning.com/books/the-creative-programmer

   - https://www.manning.com/books/the-programmers-brain

   - https://www.amazon.com/Understanding-Software-Addison-Wesley...

   I also find books about specific technologies that indeed run the risk of being deprecated after a few years to be useful too

   - https://www.oreilly.com/library/view/networking-and-kubernet...

   - https://www.brendangregg.com/bpf-performance-tools-book.html

   Furthermore, nothing keeps you from reading books about topics peripheral to computer science, say to keep up with the general vibes:

   - https://www.amazon.com/Probabilistic-Machine-Learning-Introd...

   - https://www.amazon.com/Deep-Learning-Foundations-Christopher...

   - https://www.amazon.com/Joy-Abstraction-Exploration-Category-...

   I find that all of these contribute significantly to my growth as an engineer.

   Show morethread
3. AthebJul 2024
   Not sure if my experience is relevant, but I did a couple of internships in web dev during my bachelors degree in CS and quickly realized it wasn't for me. I then did a masters and now a PhD in medical imaging where I extensively use machine learning (design and train my own models, doing both supervised and RL) but I wouldn't say I am a researcher in AI/ML.

   Because I am still in the academic process, I had the opportunity to take a couple of classes on the subject. Three books that I would recommend going over to make sure your foundation in ML and mathematics are solid are

   -Pattern recognition and machine learning by Christopher Bishop

   -Mathematics for Machine Learning by Peter Deisenroth

   -Deep Learning by Courville, Bengio and Goodfellow

   All three are legally available online in some form. I can't say I have any experience in finding a job related to ML though.

   Show morethread
4. breadwinnerDec 2023
   Here it is on Amazon: https://www.amazon.com/Deep-Learning-Foundations-Christopher...

   Now that Amazon has competition they are not offering even $1 of discount. In the old days you'd get at least $20 off list price.

   thread