Anyscale
3 HN mentions · mentioned in 3 months, 2023
18:22 · 6K views · Nov 2, 2022
A Ray Summit 2022 keynote fireside chat between OpenAI's Greg Brockman and Anyscale's Robert Nishihara. They discuss how OpenAI's training infrastructure evolved and its use of Ray.
Cited as a source for the fact that OpenAI uses Ray for training, and for Brockman's views on developer speed.
3 comments from Hacker News
I'm one of the Ray developers, thanks for the shoutout :) If you're curious about how Ray is used for LLMs, here are some interesting examples of LLM projects using Ray! - Alpa does training and serving with 175B parameter models [LINK] - GPT-J [LINK] - Another HN thread on training LLMs with Ray (on TPUs in this case) [LINK] - OpenAI fireside chat on the evolution of their infrastructure and usage of Ray for training this video - Cohere on their architecture for training LLMs [LINK] Some other thoughts 1. There is a lot more we want to do to make Ray better for working with large language models and for making training, serving, and batch inference work well out of the box. 2. The original post is about training, but we actually see even more interest in fine-tuning and serving with LLMs, in part because there are good pre-trained models. 3. For LLMs, we see a lot of interest in Ray + Jax or Ray + TPUs relative to what we see in other use cases.
Consistently I see this advice from everyone who I consider has achieved outside success in their field. Greg Brockman (OpenAI Cofounder) : Developer Speed is the most important thing I care about: ( this video ). I believe he said more about this topic but I can't find it now. Frank Slootman (Snowflake CEO): In his book Amp It Up, he stresses why pushing people to move fast is extremely important Elon Musk (Let's not get into a debate about Elon and just agree that he has helped setup 2 100bn+ companies) : "I believe in maniacal sense of urgency, if you can implement something right after the meeting, do that", upon his work philosophy when taking over twitter. In my personal experience one of the most impressive coders I met, worked extremely fast while also being very accurate, almost indicating that no tradeoff even exists.
[Author] OpenAI uses Ray, the open source version of Anyscale Platform, for training their models (video evidence: this video ). Ray ( [LINK] ) is available for anyone to use. You can also use Aviary ( [LINK] ) to serve any of those models yourself.