Sam Greydanus is a machine learning researcher who writes about physics-inspired neural networks, small-scale deep learning, and optimization.
Commenters call it a beautiful blog and single out the wing optimization and scaling-down posts as favorites.
I remember John Carlos Baez having some interesting stuff: https://johncarlosbaez.wordpress.com/
Some other ones I had bookmarked the other day but haven't seen much in depth, mostly computer science based but more towards theory/math side:
https://blog.computationalcomplexity.org/
https://semantic-domain.blogspot.com/
https://greydanus.github.io/
Also go to Terry Tao's blog and look at the sidebar, there are a lot of links to external blogs there.
I found the previous Hamiltonian Neural Networks [0]. If the authors are here, I'd be interested to use [1] for a version with dissipation.
[0] https://greydanus.github.io/2019/05/15/hamiltonian-nns/
[1] https://arxiv.org/abs/1902.04598
Some of the older posts are very cute. I enjoyed https://greydanus.github.io/2020/12/01/scaling-down/ on showing how many high-powered tricks you can do with a tiny NN of the sort that will train in seconds.
This is somewhat related: how do we build datasets that are better discriminators of model architectures, and are also easier to experiment on?
https://greydanus.github.io/2020/12/01/scaling-down/
This is a beautiful blog, e.g. this one is one of my favorites https://greydanus.github.io/2020/10/14/optimizing-a-wing/
Original source, https://greydanus.github.io/2017/01/07/enigma-rnn/