Epoch AI

Wait hold on, let's put some numbers on this. Please correct my calculations if I'm wrong.

1. The human brain draws 12 - 20 watts [1, 2]. So, taking the lower end, a task taking one hour of our time costs 12 Wh.

2. An average ChatGPT query is between 0.34 Wh - 3 Wh. A long input query (10K tokens) can go up to 10 Wh. [3] I get the best results by carefully curating the context to be very tight, so optimal usage would be in the average range.

3. I have had cases where a single prompt has saved me at least an hour of work (e.g. https://news.ycombinator.com/item?id=44892576). Let's be pessimistic and say it takes 3 prompts at 3 Wh (9 Wh) and 10 minutes (2 Wh) of my time prompting and reviewing to complete a task. That is 11 Wh for the same task, which still beats out the human brain unassisted!

And that's leaving aside the recent case where I vibecoded and deployed a fully-tested endpoint on a cloud platform I had no prior experience in, over the course of 2 - 3 hours. I estimate it would have taken me a whole day just to catch up on the documentation and another 2 days tinkering with the tools, commands and code. That's at least an 8x power savings assuming an 8-hour workday!!

4. But let's talk data instead of anecdotes. If you do a wide search, there is a ton of empirical evidence that improves programmer productivity by 5 - 30% (with a lot of nuance). I've cited some here: https://news.ycombinator.com/item?id=45379452 -- there is no measure of the amount of prompt usage…

If one wants to follow AI development mostly in the sense of LLMs and associated frontier models, that's an excellent list with over half of the names familiar, to whom I have converged independently.

I have a list in X for AI; it's the best source of information overall on the subject, although some podcasts or RSS feeds directly from the long-form writers would be quite close. (If one is a researcher themselves, then of course it's a must to follow the paper feeds, not commentary or secondary references.)

I'd add https://epoch.ai to the list, on podcasts at least Dwarkesh Patel; on blogs Peter Wildeford (a superforecaster), @omarsar0 aka elvis from DAIR in X, also many researchers directly although some of them like roon or @tszzl are more entertaining than informative.

The point about polluted information environment resonates on me; in general but especially with AI. You get a very incomplete and strange understanding by following something like NYT who seem to concentrate more on politics than technology itself.

Of course there are adjacent areas of ML or AI where the sources would be completely different, say protein or genomics models, or weather models, or research on diffusion, image generation etc. The field is nowadays so large and active that it's hard to grasp everything that is happening on the surface level.

Do you _have_ to follow? Of course not, people over here are just typically curious and willing to follow groundbreaking technological advancements.…

The source here is "CAISI Evaluation of DeepSeek V4 Pro" [1]; the US NIST ran their own benchmarks (including several internal ones) and reported the following table:

| Domain | Benchmark | Model (reasoning level) | | | |
|--:-------------------|------------------------|-------------------------|-----------------------------|--------------------------|-----------------------|
| | | OpenAI GPT-5.5 (xhigh) | OpenAI GPT-5.4 mini (xhigh) | Anthropic Opus 4.6 (max) | DeepSeek V4 Pro (max) |
| Cyber | CTF-Archive-Diamond | **71%** | 32% | 46% | 32% |
| Software Engineering | SWE-Bench Verified* | **81%** | 73% | 79% | 74% |
| | PortBench | **78%** | 41% | 60% | 44% |
| Natural Sciences | FrontierScience | **79%** | 74% | 72% | 74% |
| | GPQA-Diamond | **96%** | 87% | 91% | 90% |
| Abstract Reasoning | ARC-AGI-2…

A million people like you can do a million times more things that require intelligence†, and would consume a million times more Taco Bell double deckers. Ergo, in that limited sense, more energy == more intelligence.

"Over time" clearly means that he's talking about the far future, not the next few years when they're cleaning up the low-hanging fruit. Rather than "absurd ... fantastic and impossible" I would describe the Dyson-sphere outcome as inevitable, unless humanity goes extinct within a few centuries. Maybe you thought he meant next March?

In https://epoch.ai/gradient-updates/how-much-energy-does-chatgpt-use, Josh You, Alex Erben, and Ege Erdil estimate that a typical GPT-4o query uses 0.3 watt hours, based on the estimates that it's a 400-billion-parameter MoE model in which ¼ of the parameters are activated for a given query, so each token requires 200 gigaflops (100 billion multiplies and 100 billion adds I guess), a typical query produces 500 output tokens (about a page of text), and they're running on 1-petaflop/s H100 GPUs, with the overall cluster consuming at peak 1500 watts per GPU, with a utilization rate of 10%, and that the cluster on average consumes 70% of peak power. This works out to 1.05 kilojoules, or about 0.25 kilocalories, the amount of food energy in one gram of carbohydrates.

So, a 320-kcal Double DeckerⓇ Taco™ works out to 1280 GPT-4o queries answered, and a standard 2000kcal/day diet works out to 8000 GPT-4o queries per day, if we…

I find this unconvincing. The actual discussion of LLM generation is very lacking.

The original link [1] cites a discussion of the cost per query of GPT-4o at 0.3whr [2]. When you read the document [2] itself you see 0.3whr is a lower bound & 40whr is the upper bound. The paper [2] is actually pretty solid, I recommend it. It uses the public metrics from other LLM APIs to derive a likely distribution of the context size of the average query for GPT-4o which is a reasonable approach given that data isn't public. Then factoring in GPU power per FLOP, average utilization during, and cloud/renting overhead. It admits this likely has non-trivial error bars, concluding the average is between 1-4whr per query.

This is disappointing to me as the original link [1] attempts to bring in this source [2] to disprove the 3whr "myth" created by another paper [3], yet this 3whr figure lies directly in the error bars their new source [2] arrives at.

Links:

1. https://simonwillison.net/2025/Apr/29/chatgpt-is-not-bad-for-the-environment/

2. https://epoch.ai/gradient-updates/how-much-energy-does-chatgpt-use

3. https://www.sciencedirect.com/science/article/pii/S2542435123003653?dgcid=author

Edit: whr not w/hr

I obviously don't know the full economics of the Chinese-hosted models, but estimates[1] put the cost of hardware (servers + networking) at 70-80% of the total cost. Those things aren't meaningfully cheaper in China, so serving DeepSeek at 1/3 the cost of the cheapest US provider doesn't really compute unless it's heavily subsidized or we believe that Chinese engineers are just that much better at optimization.

Edge models, yes, they can be convenient to run batch jobs locally. I still would argue there's no economic benefit over paying for models. Haiku has a bad price/perf but others in that class are significantly cheaper in hosted APIs.

Doesn't matter what I think, the reality is that the majority of enterprises (where the real $ comes from) will not consider sending their data to China.

1. https://epoch.ai/data-insights/ai-datacenter-cost-breakdown