Dwarkesh Patel

Efficient markets route around bottlenecks. Technological revolutions accelerate the speed at which that re-routing happens.

In software, we, the developers, have increasingly been a bottleneck. The world needs WAY more software than we can economically provide, and at long last a technology has arrived that will help route around us for the benefit of humanity.

Here's an excellent Casey Handmer quote from a recent Dwarkesh episode:

> One way to think about the industrial revolutions is [...] what you're doing is you're finding some way of bypassing a constraint or bypassing a bottleneck. The bottleneck prior to what we call the Industrial Revolution was metabolism. How much oats can a human or a horse physically digest and then convert into useful mechanical output for their peasant overlord or whatever? Nowadays we would giggle to think that the amount of food we produce is meaningful in the context of the economic power of a particular country. Because 99% of the energy that we consume routes around our guts, through the gas tanks of our cars and through our aircraft and in our grids and stuff like that.

> Right now, the AI revolution is about routing around cognitive constraints, that in some ways writing, the printing press, computers, the Internet have already allowed us to do to some extent. A credit card is a good example of something that routes around a cognitive constraint of building a network of trust. It's a centralized trust.

It's a great episode, I…

This is a super interesting dynamic! The CCP is really good at subsidizing and flooding global markets, but in the end, it takes power to generate tokens.

In my Uber comparison, it was physical hardware on location... taxis, but this is not the case with token delivery.

This is such a complex situation in that regard, however, once the market settles and monopolies are created, eventually the price will be what market can bear. Will that actually create an increase in gross planet product, or will the SOTA token providers just eat up the existing gross planet product, with no increase?

I suppose whoever has the cheapest electricity will win this race to the bottom? But... will that ever increase global product?

___

Upon reflection, the comment above was very likely influenced by this truly amazing quote from Satya Nadella's interview on the Dwarkesh podcast. This might be one of the most enlightened things that I have ever heard:

> Us self-claiming some AGI milestone, that's just nonsensical benchmark hacking to me. The real benchmark is: the world growing at 10%.

https://www.dwarkesh.com/p/satya-nadella#:~:text=Us%20self%2Dclaiming%20some%20AGI%20milestone%2C%20that%27s%20just%20nonsensical%20benchmark%20hacking%20to%20me.%20The%20real%20benchmark%20is%3A%20the%20world%20growing%20at%2010%25.

Do you think it's akin to Ilya's [1] claim that next token prediction is reality? E.g. any deeper claims about the structure of that intelligence or comparing to humans?

To be clear, I'm 100% with you that "next token predictor" is stupid to call what these machines are now. We are engineers and can shape the capability landscape to give rise to a ton of emergent behavior. It's kind of amazing. In that sense, being precise about what's going on, rather than being essentialist (technically, yes, the 'actual' algorithm, whatever that even means, is text prediction), is just good epistemology.

I still think it's still a very interesting question though to ask about deeper emergent structures. To me, this is evidence of a more embedded cognition kind of theory of intelligence (admittedly this is not very precise). But IDK how into philosophy you are.

[1] https://www.dwarkesh.com/p/ilya-sutskever

> It’s not “free tier” it’s all the tiers being significantly discounted (underpriced). It’s ALL pricing.

That's possible, but how would we know what the cost to provide the service is from any of the companies, which will anyways vary from company to company since they are using different models and hardware (e.g. OpenAI/Microsoft NVIDIA, Google TPUs, Anthropic/Amazon Trainium).

We also don't know how much the average chat subscriber is using the service. A low usage subscriber may be profitable at $20/mo, but a high usage one unprofitable.

Chat subscription prices seem to have been steady at $20/mo for a while, but cost of providing the service is falling very fast. In recent Dwarkesh interview, Satya Nadella says costs are falling 5/10/40x year over year, or evern quarter over quarter.

Here, at 1:10:56

https://www.dwarkesh.com/p/satya-nadella-2

I thought Dwarkesh's position was pretty thoughtful: https://www.dwarkesh.com/p/dow-anthropic

> Nobody is qualified to steward the development of superintelligence. It is a terrifying, unprecedented thing that our species is doing right now, and the fact that private companies aren’t the ideal institutions to take up this task does not mean the Pentagon or the White House is.

>
The only way we can preserve our free society is if we make laws and norms through our political system that it is unacceptable for the government to use AI to enforce mass surveillance and censorship and control. Just as after WW2, the world set the norm that it is unacceptable to use nuclear weapons to wage war.

Zuckerberg explains a few of the reasons here:

https://www.dwarkesh.com/p/mark-zuckerberg#:~:text=As%20long%20as%20it%27s%20helping%20us%20then%20yeah.

The short version is that is you give a product to open source, they can and will donate time and money to improving your product, and the ecosystem around it, for free, and you get to reap those benefits. Llama has already basically won that space (the standard way of running open models is llama.cpp), so OpenAI have finally realized they're playing catch-up (and last quarter's SOTA isn't worth much revenue to them when there's a new SOTA, so they may as well give it away while it can still crack into the market)