Lambda Labs Blog (lambdalabs.com)

What HN people wrote when they linked it

refulgentisMar 2024
I'm waiting for a build so I went fully down this trail. XD
A1) Yes it can be done with a 4090. (2a).
A2) 2 days. (4d/4e).
B) Up to you, author did both and settled on 50M which they got to 1300 ELO. (note: they also did subsequent work with the same model to increase perf without further training) (2a).
C) Mu: there's no limit as you can page in/out (3a, 3c), and its trivial to store in both RAM and VRAM. 30B fits in memory with 32 GB of RAM or VRAM. (3e)
Starting from:
1. article: https://adamkarvonen.github.io/machine_learning/2024/03/20/chess-gpt-interventions.html
2. second sentence links to post on training: https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html.
2a. "A 50 million parameter GPT trained on 5 million games of chess learns to play at ~1300 Elo in one day on 4 RTX 3090 GPUs"
2b. "The 50M parameter model played at 1300 Elo with 99.8% of its moves being legal within one day of training"
3. re: most parameters 4090 can do:
3a. understanding is there is no limitation, in that, you don't _need_ to have every parameter in memory at all times, either during training or inference.
3b. google "are amount of parameters in llm limited by vram size"
3c. go to /r/LocalLLaMa link: https://www.reddit.com/r/LocalLLaMA/comments/15j0mvm/what_are_the_variables_to_take_into_account_to/ (why? its a favorite of mine, believe its the closest you get to local training ppl talking in open space that's not discord)
3d. understanding in 3a…
charlesllMar 2021
Lambda Labs|Director of Marketing|Remote|https://lambdalabs.com/
About the Role:
Lambda provides GPU infrastructure to the world’s leading AI labs. As our Director of Marketing you’ll lead everything from product marketing to demand generation to reach more AI engineers and researchers. Our business has grown by more than 10x over the last few years and we’re looking for a marketing leader who can help take us through our next stage of growth. Reporting to our CEO, you’ll have complete end to end ownership of our digital ad campaigns, setting our marketing strategy, budget, and generating marketing qualified leads.
About You:
Strong technical marketing background, ideally in enterprise software or hardware. Years of experience managing large demand generation budgets and crafting.
impactful product messaging. Proven management experience.
Experience We're Looking For:
-5+ years of experience in a management role.
-Extensive experience targeting a hardware and/or SaaS audience.
-Startup experience & mindset with a bias for action.
-Metrics and data-driven approach.
-Able to operate independently and cross functionally to build out new programs and initiatives.
-Deep understanding of CRM, marketing automation, digital marketing and email marketing tactics and tools.
-Excellent written and verbal communication skills.
-BS EECS/Math or previous work experience in an engineering position.
https://boards.greenhouse.io/lambda/jobs/4372122003
Contact:Charles…
pbronezSep 2022
Lambda Labs crunched the numbers in Feb 2022 [0]. They concluded:
“””
So, which GPUs to choose if you need an upgrade in early 2022 for Deep Learning? We feel there are two yes/no questions that help you choose between A100, A6000, and 3090. These three together probably cover most of the use cases in training Deep Learning models:
Do you need multi-node distributed training? If the answer is yes, go for A100 80GB/40GB SXM4 because they are the only GPUs that support Infiniband. Without Infiniband, your distributed training simply would not scale. If the answer is no, see the next question.
How big is your model? That helps you to choose between A100 PCIe (80GB), A6000 (48GB), and 3090 (24GB). A couple of 3090s are adequate for mainstream academic research. Choose A6000 if you work with a large image/language model and need multi-GPU training to scale efficiently. An A6000 system should cover most of the use cases in the context of a single node. Only choose A100 PCIe 80GB when working on extremely large models
“””
[0] https://lambdalabs.com/blog/best-gpu-2022-sofar/
tbenstSep 2020
Thank you for cross-referencing that, the data does look accurate and my statement now seems exaggerated. I do think we need skepticism on the A100 charts though until third party benchmarks.
> In my experience NVidia benchmark numbers in deep learning are rarely lies - they are highly optimised, in optimal conditions and rarely achievable in the real world.
Right, but Nvidia claimed 1360 images/sec for resnet-50 on imagenet. To my knowledge this still hasn’t been realized by a third party. It also isn’t a 4x improvement for fp32 vs fp16, that’s comparing to previous generation. Improvement is more like 1.5x: https://lambdalabs.com/blog/best-gpu-tensorflow-2080-ti-vs-v100-vs-titan-v-vs-1080-ti-benchmark/
Even in very simple synthetic benchmarks the speed up is only 2x: https://github.com/tensorflow/benchmarks/issues/77#issuecomment-349838985
I have not seen any benchmarks showing an 8x speedup. Have you? If not -> Nvidia lied.
sabalabaJul 2019
Lambda | Linux Technical Support Specialist | San Francisco, CA (Dogpatch) | Onsite | Full Time | https://lambdalabs.com
We're looking for a linux technical support specialist who wants to help provide our customers with world class UNIX/Linux support for their GPU servers, workstations, laptops, and cloud instances.
* Be first point of contact for all incoming technical support questions (phone, email, chat, and forum)
* Troubleshoot software and hardware issues for customers
* Document solutions to common problems
Required Experience:
* Linux background, hobbyist or professional
* Shell and Python scripting proficiency
* Excellent written and oral communication skills
===
We're located in San Francisco 5 minutes away from the 22nd street Caltrain.
Apply to: https://lambdalabs.com/careers
NBJackAug 2023
I'm not sure I follow. Knowledge based retrieval of what exactly? Outdated docs and dilapidated code? Aging internal wikis and chat histories erased for legal reasons?
Everyone also seems to overlook how much time and resources it takes for these models to be trained/fine tuned on a corpus of knowledge. Researches have calculated it probably took OpenAI the equivalent of 355 years on a single NVidia V100 to train up GPT 3. [1] Clearly they used more horsepower in parallel, which is a foreseen problem right now for other reasons. [2]
[1] https://lambdalabs.com/blog/demystifying-gpt-3
[2] https://www.pcworld.com/article/2020375/the-ai-boom-could-create-a-new-gpu-shortage.html

What HN people wrote when they linked it

refulgentisMar 2024
I'm waiting for a build so I went fully down this trail. XD
A1) Yes it can be done with a 4090. (2a).
A2) 2 days. (4d/4e).
B) Up to you, author did both and settled on 50M which they got to 1300 ELO. (note: they also did subsequent work with the same model to increase perf without further training) (2a).
C) Mu: there's no limit as you can page in/out (3a, 3c), and its trivial to store in both RAM and VRAM. 30B fits in memory with 32 GB of RAM or VRAM. (3e)
Starting from:
1. article: https://adamkarvonen.github.io/machine_learning/2024/03/20/chess-gpt-interventions.html
2. second sentence links to post on training: https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html.
2a. "A 50 million parameter GPT trained on 5 million games of chess learns to play at ~1300 Elo in one day on 4 RTX 3090 GPUs"
2b. "The 50M parameter model played at 1300 Elo with 99.8% of its moves being legal within one day of training"
3. re: most parameters 4090 can do:
3a. understanding is there is no limitation, in that, you don't _need_ to have every parameter in memory at all times, either during training or inference.
3b. google "are amount of parameters in llm limited by vram size"
3c. go to /r/LocalLLaMa link: https://www.reddit.com/r/LocalLLaMA/comments/15j0mvm/what_are_the_variables_to_take_into_account_to/ (why? its a favorite of mine, believe its the closest you get to local training ppl talking in open space that's not discord)
3d. understanding in 3a…
charlesllMar 2021
Lambda Labs|Director of Marketing|Remote|https://lambdalabs.com/
About the Role:
Lambda provides GPU infrastructure to the world’s leading AI labs. As our Director of Marketing you’ll lead everything from product marketing to demand generation to reach more AI engineers and researchers. Our business has grown by more than 10x over the last few years and we’re looking for a marketing leader who can help take us through our next stage of growth. Reporting to our CEO, you’ll have complete end to end ownership of our digital ad campaigns, setting our marketing strategy, budget, and generating marketing qualified leads.
About You:
Strong technical marketing background, ideally in enterprise software or hardware. Years of experience managing large demand generation budgets and crafting.
impactful product messaging. Proven management experience.
Experience We're Looking For:
-5+ years of experience in a management role.
-Extensive experience targeting a hardware and/or SaaS audience.
-Startup experience & mindset with a bias for action.
-Metrics and data-driven approach.
-Able to operate independently and cross functionally to build out new programs and initiatives.
-Deep understanding of CRM, marketing automation, digital marketing and email marketing tactics and tools.
-Excellent written and verbal communication skills.
-BS EECS/Math or previous work experience in an engineering position.
https://boards.greenhouse.io/lambda/jobs/4372122003
Contact:Charles…
pbronezSep 2022
Lambda Labs crunched the numbers in Feb 2022 [0]. They concluded:
“””
So, which GPUs to choose if you need an upgrade in early 2022 for Deep Learning? We feel there are two yes/no questions that help you choose between A100, A6000, and 3090. These three together probably cover most of the use cases in training Deep Learning models:
Do you need multi-node distributed training? If the answer is yes, go for A100 80GB/40GB SXM4 because they are the only GPUs that support Infiniband. Without Infiniband, your distributed training simply would not scale. If the answer is no, see the next question.
How big is your model? That helps you to choose between A100 PCIe (80GB), A6000 (48GB), and 3090 (24GB). A couple of 3090s are adequate for mainstream academic research. Choose A6000 if you work with a large image/language model and need multi-GPU training to scale efficiently. An A6000 system should cover most of the use cases in the context of a single node. Only choose A100 PCIe 80GB when working on extremely large models
“””
[0] https://lambdalabs.com/blog/best-gpu-2022-sofar/
tbenstSep 2020
Thank you for cross-referencing that, the data does look accurate and my statement now seems exaggerated. I do think we need skepticism on the A100 charts though until third party benchmarks.
> In my experience NVidia benchmark numbers in deep learning are rarely lies - they are highly optimised, in optimal conditions and rarely achievable in the real world.
Right, but Nvidia claimed 1360 images/sec for resnet-50 on imagenet. To my knowledge this still hasn’t been realized by a third party. It also isn’t a 4x improvement for fp32 vs fp16, that’s comparing to previous generation. Improvement is more like 1.5x: https://lambdalabs.com/blog/best-gpu-tensorflow-2080-ti-vs-v100-vs-titan-v-vs-1080-ti-benchmark/
Even in very simple synthetic benchmarks the speed up is only 2x: https://github.com/tensorflow/benchmarks/issues/77#issuecomment-349838985
I have not seen any benchmarks showing an 8x speedup. Have you? If not -> Nvidia lied.
sabalabaJul 2019
Lambda | Linux Technical Support Specialist | San Francisco, CA (Dogpatch) | Onsite | Full Time | https://lambdalabs.com
We're looking for a linux technical support specialist who wants to help provide our customers with world class UNIX/Linux support for their GPU servers, workstations, laptops, and cloud instances.
* Be first point of contact for all incoming technical support questions (phone, email, chat, and forum)
* Troubleshoot software and hardware issues for customers
* Document solutions to common problems
Required Experience:
* Linux background, hobbyist or professional
* Shell and Python scripting proficiency
* Excellent written and oral communication skills
===
We're located in San Francisco 5 minutes away from the 22nd street Caltrain.
Apply to: https://lambdalabs.com/careers
NBJackAug 2023
I'm not sure I follow. Knowledge based retrieval of what exactly? Outdated docs and dilapidated code? Aging internal wikis and chat histories erased for legal reasons?
Everyone also seems to overlook how much time and resources it takes for these models to be trained/fine tuned on a corpus of knowledge. Researches have calculated it probably took OpenAI the equivalent of 355 years on a single NVidia V100 to train up GPT 3. [1] Clearly they used more horsepower in parallel, which is a foreseen problem right now for other reasons. [2]
[1] https://lambdalabs.com/blog/demystifying-gpt-3
[2] https://www.pcworld.com/article/2020375/the-ai-boom-could-create-a-new-gpu-shortage.html

Lambda Labs Blog

Top posts on HN

What HN people wrote when they linked it

Lambda Labs Blog

Top posts on HN

What HN people wrote when they linked it

Top posts on HN

What HN people wrote when they linked it

Related

The Missing Nvidia GPU Glossary

Google Titans architecture, helping AI have long-term memory

Volta: Advanced Data Center GPU

Ethereum: Rise of the World Computer

Aiter: AI Tensor Engine for ROCm

Simon Peyton Jones Elected into the Royal Society Fellowship

Google Duplex: An AI System for Accomplishing Real World Tasks Over the Phone

Learning Reinforcement Learning, with Code, Exercises, and Solutions

Top posts on HN

What HN people wrote when they linked it

Related

The Missing Nvidia GPU Glossary

Google Titans architecture, helping AI have long-term memory

Volta: Advanced Data Center GPU

Ethereum: Rise of the World Computer

Aiter: AI Tensor Engine for ROCm

Simon Peyton Jones Elected into the Royal Society Fellowship

Google Duplex: An AI System for Accomplishing Real World Tasks Over the Phone

Learning Reinforcement Learning, with Code, Exercises, and Solutions