BentoML, a company building tools for serving AI models, blogs about LLM inference, deployment, and open source models.
HN commenters ask how BentoML compares to other tools and link its posts on DeepSeek-OCR and inference benchmarks, one disputing its claims.
This is really not true my friend. I would love to help you if I had some more time, but let me look for a tutorial.
Because the FineWeb Dataset is already super good. You can train 7B or 32B Param models at home
The >600B Param model isn't really using all the data effectively, but with a MacStudio Farm you can also train that one at home (if you have enough money to buy at least 100).
Here's the easy way: https://github.com/FareedKhan-dev/train-deepseek-r1
More details: https://www.bentoml.com/blog/the-complete-guide-to-deepseek-models-from-v3-to-r1-and-beyond
Here's how DeepSeek-R1-Zero was built, basically from 0 to Hero, including weights the FULL Training Data and everything you need to get it running locally or on servers.https://medium.com/@GenerationAI/how-deepseek-r1-zero-was-reproduced-in-30-4e394cd3dd58
For $30 USD you can also train a small DeepSeek at home!
https://github.com/Jiayi-Pan/TinyZero
https://huggingface.co/deepseek-ai/DeepSeek-R1-Zero (the model)
> What about alternatives?
Custom models? Apart from the Big 3 (in no particular order):
- https://together.ai/
- https://replicate.com/
- https://anyscale.com/
- https://baseten.co/
- https://modal.com/
- https://banana.dev/
- https://runpod.io/
- https://bentoml.com/
- https://brev.dev/
- https://octo.ai/
- https://cerebrium.ai/
...
There's also the option of platforms such as BentoML (I have no affiliation) that offer usage-based pricing so you can at least take the 100% utilization assumption off the table. I'm not sure how the price compares to EKS.
https://www.bentoml.com/
There are already several fully open source LLMs. You can start participating in those projects today.
https://www.bentoml.com/blog/navigating-the-world-of-open-source-large-language-models#what%20is%20the%20best%20open-source%20llm%20now%3F
Seems there’s still some confusion around what DeepSeek-OCR really does. Learn about the model, Contexts Optical Compression, and its impact on LLMs here: https://www.bentoml.com/blog/deepseek-ocr-contexts-optical-compression-explained
Is it really that close in production? The benchmarks I've seen show it's almost 3x slower compared to the SOTA. https://bentoml.com/blog/benchmarking-llm-inference-backends