Pete Warden

Is this surprising? I think it's expected, when most published numbers from Google suggest that this is only 2x over DistBelief, which Adam Coates improved upon 6x (http://www.cs.stanford.edu/~acoates/papers/CoatesHuvalWangWuNgCatanzaro_icml2013.pdf) in 2013. Coming from a HPC background, DistBelief (and sometimes MR in general) has always felt like a square being forced into a round hole. Sure it "scales" but it's not how you would build a system from scratch to meet the same computational needs. Moving data around is much slower than doing extra math.

DNNs provide some interesting opportunities for domain specific optimization with lossy math because empirically it seems that the solutions dont require significant precision. Something like a general Tensor library holds you back from using some of the tricks like synchronization-free hogwild, where it wouldn't be appropriate in other scenarios where more exact computation is necessary.

One interesting step towards this is their implementation of lossy compression with a custom 16-bit half float representation. However a lot of people are dialing down the precision even further, including one of the authors of TensorFlow (http://petewarden.com/2015/05/23/why-are-eight-bits-enough-for-deep-neural-networks/).

Scott Gray, one of the principle engineers at Nervana Systems, has a very wise reply in the github thread - they aren't doing much to optimize for memory locality, and wont have amazing speedups until that happens.

For deep learning or chips for deep learning?
With an EE/Chem/Math background you should be set to go :)

For deep learning, I highly recommend the cs231n course materials (available for free online) and the Deep Learning Book by Goodfellow et al.

For chip design for deep learning specifically, it's a fairly new field, and a lot of commercial interests, hence why a lot of it isn't available in the form of an "Intro to Deep Learning Chip Design" course. But, the basic point is that deep learning can both train and perform inference in astonishingly low precision.

For training: https://arxiv.org/abs/1502.02551

For inference, there are so many papers confirming this fact that there really isn't one key paper to point to. This article is a good introduction however: https://petewarden.com/2015/05/23/why-are-eight-bits-enough-for-deep-neural-networks/

Other than that, stripping out the cache hierarchy, HBM memory, etc. are the obvious steps to take in targeting deep learning. Note that it's my opinion that deep learning chip startups (such as ours) need another "secret sauce" beyond simply lowering precision, we're not like Intel which can just say "okay, here's $100mil, make me an 8-bit GPU, Go."

Integration. RPI is nice, but it has a bunch of things that are off spec for the purpose (Pretty decent 3D graphics, but really painful to do GPU work on http://petewarden.com/2014/08/07/how-to-optimize-raspberry-pi-code-using-its-gpu/ and GPU programming is a bit advanced for 11 year olds anyways), and lacking things that allow a diverse book of "labs" to be done in class. Having built in sensors and outputs is critical. If a school had to outfit their computer labs (to the extent they still even have them) with extra HDMI screens for every work station it would really drive up the price of the program, and limit the ability.

This isn't supposed to be powerful, it is supposed to be an integrated tool that allows reasonably fun labs and learning sections for a diverse group of kids. You could arguably do the same with shields and arduino, but then you lose the form factor.

Cool tutorial! It's great to see examples of people building models that are designed to be run in real time on the Raspberry Pi.

This post reminds me that I really need to find time to update the RPi TensorFlow Github repo. The TensorFlow team is making progress on official support- check out their nightly binaries here:

Model 2/3 B:

Python 2.7: http://ci.tensorflow.org/view/Nightly/job/nightly-pi/
Python 3.4: http://ci.tensorflow.org/view/Nightly/job/nightly-pi-python3/

Pi Zero:

Python 2.7: http://ci.tensorflow.org/view/Nightly/job/nightly-pi-zero/
Python 3.4: http://ci.tensorflow.org/view/Nightly/job/nightly-pi-zero-python3/

Github discussion: https://github.com/tensorflow/tensorflow/issues/5729
Pete Warden's blog post on cross-compiling: https://petewarden.com/2017/08/20/cross-compiling-tensorflow-for-the-raspberry-pi/

Yes, your comment was a paragon of unemotional wisdom. You have no interest here other than cool exercise of rational thought. You aren't upset at all. It's those other people who have let their feelings run away with them. Just keep telling yourself that. Because as we know, feelings like empathy and concern have no place in a discussion about how humans should treat other humans.

Funnily, I agree with you that nerds are behaving badly now that they have power. [1] But I disagree with your conclusion. What happened to Sierra is a great example of nerds misusing their sudden social power. Now we need to start taking responsibility for the broader effects of our works and our previously insular culture.

[1] For others, this is a good analysis: http://petewarden.com/2014/10/05/why-nerd-culture-must-die/

There is also the much larger confusion arising of whether it's an ML compiler in the sense of being a specialized compiler for ML code (compiles a description of a network to efficient ML-specific low-level IR to enable further optimization and code generation), or whether it's an ML compiler, a compiler (or part of it) that uses ML.

The first meaning is extremly common, the top 2 results for "ML Compiler" on google returns[1][2], both using it in the first sense. It's not that ML or ML-like techniques in compiler writing is new but the first sense is definitely at least as popular an interpretation.

[1] https://huyenchip.com/2021/09/07/a-friendly-introduction-to-machine-learning-compilers-and-optimizers.html

[2] https://petewarden.com/2021/12/24/why-are-ml-compilers-so-hard/