ChinaTalk

This interview with DeepSeek founder and CEO Liang Wenfeng, also co-founder of the hedge fund backing DeepSeek, might shed some light on the question: https://www.chinatalk.media/p/deepseek-ceo-interview-with-chinas

Some relevant excerpts:

“Because we believe the most important thing now is to participate in the global innovation wave. For many years, Chinese companies are used to others doing technological innovation, while we focused on application monetization — but this isn’t inevitable. In this wave, our starting point is not to take advantage of the opportunity to make a quick profit, but rather to reach the technical frontier and drive the development of the entire ecosystem.”

“We believe that as the economy develops, China should gradually become a contributor instead of freeriding. In the past 30+ years of the IT wave, we basically didn’t participate in real technological innovation. We’re used to Moore’s Law falling out of the sky, lying at home waiting 18 months for better hardware and software to emerge. That’s how the Scaling Law is being treated.

“But in fact, this is something that has been created through the tireless efforts of generations of Western-led tech communities. It’s just because we weren’t previously involved in this process that we’ve ignored its existence.”

“We do not have financing plans in the short term. Money has never been the problem for us; bans on shipments of advanced chips are the problem.”

“In the face of disruptive…

> A bunch of ML researchers who were initially hired to do quant work

Very interesting! I'm sure you have a source for this claim?

This myth of DS being a side project literally started from one tweet. DeepSeek the company is funded by a company whose main business is being a hedge fund, but DeepSeek itself from day 1 has been all about building LLM to reach AGI, completely independent.

This is like saying SpaceX is the side-project of a few caremaking bros, just because Elon funded and manages both. They are unrelated.

Again, you can easily google the name of the authors and look at their background, you will find people with PhD in LLM/multimodal models, internships at Microsoft Research etc. No trace of background on quant or time series prediction or any of that.

From the mouth of the CEO himself 2 years ago: "Our large-model project is unrelated to our quant and financial activities. We’ve established an independent company called DeepSeek, to focus on this." [0]

It's really interesting to see how after 10 years debating the mythical 10x engineer, we have now overnight created the mythical 100x Chinese quant bro researcher, that can do 50x better models than the best U.S. people, after 6pm while working on his side project.

[0]: https://www.chinatalk.media/p/deepseek-from-hedge-fund-to-frontier

>Speaking at a Council on Foreign Relations event on Monday, Amodei said that China is known for its “large-scale industrial espionage” and that AI companies like Anthropic are almost certainly being targeted.

Also from https://www.chinatalk.media/p/anthropics-dario-amodei-on-ai-competition

> Jordan Schneider: We’ve talked about competition from the lens of compute, from the lens of models, but models are created by people. The times I’ve been to NeurIPS, it has been really striking to me just how much Mandarin is spoken at those poster sessions. I’m curious what message you’d have to say to PRC nationals studying in the West who hear this and are like, “Why would I want to work for this guy?"

> Dario Amodei: I want to be really clear on one thing — I should probably say it even more than I do. When we talk about China, this isn’t about Chinese people versus American people.

But we'll totally suspect you of being a spy for the PRC

The article says it had to be open source because otherwise people wouldn't trust the Chinese but ByteDance, Tencent, Baidu, and Alibaba also do LLMs and are not open source.

It's funny reading an article interviewing the ceo:

>Until now, among the seven major Chinese large-model startups, it’s the only one... that hasn’t fully considered commercialization, firmly choosing the open-source route without even raising capital.

>While these choices often leave it in obscurity, DeepSeek frequently gains organic user promotion within the community.

The obscurity thing hasn't lasted! (article nov 2024 https://www.chinatalk.media/p/deepseek-ceo-interview-with-chinas)

The ceo's actual argument for open source is quite interesting, basically that it helps attract the best people and the value is in the team. It's kind of what used to work for OpenAI before it became the ClosedAI division of Microsoft.

I told a friend of mine this and he pointed me to an article by another friend of ours: https://www.chinatalk.media/p/a-cancer-patients-tour-of-chinese

Here's the key quote:

> The tradeoff is pretty straightforward: the US system emphasizes uniform standards and upfront rigor, while China’s [investigator-initiated trial] model pushes decision-making closer to the doctor and the patient, making it easier to start trials quickly and iterate as data comes in

It's clear that they are prioritizing iteration over standardization - which is a good pathway to exploration but will yield results like this. In the end, we're all going to benefit from the new research coming out of China as they subject themselves to this high-variance policy.

I read the article you linked:

https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens-in

I know that Anthropic claims that "large-scale coordinated "distillation attacks"" exist. They are not a disinterested party here and they don't provide evidence either.

Also you're just moving goalposts around. The article you linked also cites Anthropics claim but then paints a much much mor nuanced picture.