NVIDIA Blog

>Deep learning, modeled on the human brain, is infinitely more complex [than machine learning]. Unlike machine learning, deep learning can teach machines to ignore all but the important characteristics of a sound or image-a hierarchical view of the world that accounts for infinite variety. -Yann LeCun

I strongly disagree with a lot about this quote, even though it comes from a brilliant man I highly respect (my thesis research was inspired from some of his older work, and in my job we work with techniques he developed around deep learning). What I dislike is it utilizes hype to make deep learning seem mystical; in actuality it's a natural extension of old techniques that clearly fall under "machine learning".

Deep learning is a neural network with 3 or more layers instead of the 2 layer networks that were developed in the 1980s. People tried 3 layers back then, and they didn't work well. Yann LeCunn and other researchers found cool ways to get 3+ layers to work in the 90s and again in the mid 2000s. More recently, researchers have just thrown a ton of data and computational power at them to get them to work. But fundamentally this was an extension of established techniques. This article that recently hit the front page actually breaks it down really well: https://blogs.nvidia.com/blog/2016/07/29/whats-difference-artificial-intelligence-machine-learning-deep-learning-ai/

I think my main point here is that deep learning is quite accessible to people who are learning machine…

In 1982 you still had "supercomputers" like

https://en.wikipedia.org/wiki/Cray_X-MP

because you could still make bipolar electronics that beat out mass-produced consumer electronics. By the mid 1990s even IBM abandoned bipolar mainframes and had to introduce parallelism so a cluster of (still slower) CMOS mainframes could replace a bipolar mainframe. This great book was written by someone who worked on this project

https://campi.cab.cnea.gov.ar/tocs/17291.pdf

and of course for large scale scientific computing it was clear that "clusters of rather ordinary nodes" like the

https://www.cscamm.umd.edu/facilities/computing/sp2/index.htm

we had at Cornell were going to win (ours was way bigger) because they were scalable. (e.g. the way Cray himself saw it, a conventional supercomputer had to live within a small enough space that the cycle time was not unduly limited by the speed of light so that kind of supercomputer had to become physically smaller, not larger, to get faster)

Now for very specialized tasks like codebreaking, ASICs are a good answer and you'd probably stuff a large number of them into expansion cards into rather ordinary computers and clusters today possibly also have some ASICs for glue and communications such as

https://blogs.nvidia.com/blog/whats-a-dpu-data-processing-unit/

----

The problem I see with people who attempt parallelism for the first time is that the task size has to be smaller than the overhead to transfer tasks between cores or…

Your points are excellent, but number 2 seems a weak: why think of it as a single "machine"? If that's the metric, dual- and quad-processing machines would counter any advantage.

Plus or minus some overclocking, a GTX280 and a Core i7 960 both consume roughly the same number or watts (130 for the processor vs 200 for the card?) and cost roughly the same (at least they did when I bought comparables). So while you can stick (and probably should) stick multiple cards in the same machine, it seems a little silly measure this in terms of multiples of single CPU performance.

NVidia's blog response is interesting, and the some of the comments are actually worth reading:
http://blogs.nvidia.com/ntersect/2010/06/gpus-are-only-up-to-14-times-faster-than-cpus-says-intel.html

To me, the interesting question here is the degree to which the code has been optimized in each case, and the difficulty of doing so. A while ago there was a competition to attempt to generate SHA1 hash collision: http://www.engineyard.com/blog/2009/programming-contest-and-the-winners-are/

The GPU's trounced every other approach. I participated in the contest minimally (running slightly tweaked versions other people's code on both an Nvidia GPU and a Core i7) and I was amazed at how much more efficient the GPU solution was. I don't recall the exact ratios, but 'orders of magnitude' was definitely not an exaggeration.

But that's sort of a contrived example. So what I'd love to see here is a framework…

I am sure most at OpenAI realize that they don't have a moat[0] that will keep them ahead for an extended period of time. Being a first mover does have its advantages of course, funding and resources from MSFT also help, but if they continue to (temporarily) take away features from their paying customers, are not forthcoming with whether there have been changes to in-use models and generally make it hard to rely on and trust them to deliver a consistent service, that will further people looking for alternatives, be that other competitors[1] or local models[2]. Unless they can capture regulators in ways that would make local models impossible, they have a very narrow time window to retain a large section of the market in the medium term.

Despite their best efforts at presenting themselves as the "ethical" ones in this industry and attempts to convince governments that open/local models must be regulated[3], the only way I can see them accomplishing longer term limits on local LLMs considering the many purposes GPUs and inference accelerators can serve to the public will be hard. I have said this before, but I view this similarly to the Crypto Wars[4] or more recently Nvidia LHR[5].

I can see a Clipper Chip[6] for GPUs, that tries to limit inference to features approved for the common public, (mainly gaming focused applications like DLSS) in the near future, but I am certain that like before, where there is a 10ft wall, 12ft ladders will emerge.

Basically, OpenAI must…

>> I'm willing to bet that 99% of the performance difference will be solved by value types. Anyway, Cliff Click, who wrote H2O[1], a large machine learning platform, reports achieving Fortran speeds (i.e. maximum throughput) with pure Java[2]. This means that a C application won't even be 1% faster.

I've had a personal conversation with Cliff himself. Java no matter what you do can't deal with hardware acceleration and gpus. We agreed on that. Numerical software is a different beast. I also kept mentioning "simd instructions" as well as things like openmp.

You are talking about systems software. Unfortunately that matters a lot for machine learning. The axis along which you can get equivalent speeds should be specified here. "values types" != "runs on faster chips"

You aren't likely to beat intel or nvidia's compilers at their own game here . Java will always be playing catch up to last gen's tech there.

Disclosure: I'm more than aware of what's going on in the space. We compete with them for customers and have a very clear understanding of their offerings. H20 has a great k/v store based on the exact mechanics you're talking about. That's about it though. Also of note: Cliff doesn't work on h20 anymore: https://twitter.com/cliff_click/status/700817408110399492

>> Oh, sure, for some specialized use cases of course that's true, but you could use OpenCL in Java, too....

OpenCL isn't exactly the industry standard for this stuff. You always end up using cuda, and you…

RL can be massively disappointing, indeed. And I agree with you (and with the amazing post I already referenced [1]) that it is hard to get it to work at all. Sorry to hear you have been disappointed so much!

Nonetheless, I would personally recommend even just learning the basics and fundamentals of RL. Beyond supervised, unsupervised, and the most-recent and well-deservedly hyped semi-supervised learning (generative AI, LLMs, and so on), reinforcement learning indeed models the learning problem in a very elegant way: an agent interacting with an environment and getting feedback. Which is, arguably, a very intuitive and natural way of modeling it. You could consider backward error correction / propagation as an implicit reward signal, but that would be a very limited view.

On a positive note, RL has very practical sucessful applications today - even if in niche fields. For example, LLM fine-tuning techniques like RLHF successfully apply RL to modern AI systems, companies like Covariant are working on large robotics models which definitely use RL, and generally as a research field I believe (but I may be proven wrong!) there is so much more to explore. For example, check Nvidia Eureka that combines LLM to RL [2]: pretty cool stuff IMHO!

Far from attempting to convince you on the strength and capabilities of DRL, just recommending folks to not discard it right away and at least give it a chance to learn the basics, even just for an intellectual exercise :) Thanks…