Melanie Mitchell, an AI researcher, writes a Substack examining claims about large language models, reasoning, analogy-making, and AI risk.
Commenters cite it for careful rebuttals of AI hype, such as the claim that half of AI researchers expect a 10% chance of doom.
In an example [1] referenced in the article, ChatGPT (maybe v3.5) did a poor job of answering the following question.
Boll is a new company founded by three recent MBAs. The company plans to buy used automobiles at auctions and then sell them at four times the purchase price direct to consumers (i.e., a car that is purchased for $1,000 would be sold at $4000). Based on an analysis of similar companies, the owners of the company expect to turn their inventory 1.5 times per year. To achieve an annual revenue of $3,000,000, how much inventory, expressed in dollars, should the company expect its operation to require?
How does GPT4 fare?
------
Let's break down the problem step by step.
The company wants to achieve an annual revenue of $3,000,000.
They plan to sell cars at 4 times the purchase price.
They expect to turn their inventory 1.5 times per year.
First, we need to find out how much revenue they will generate per inventory turnover. To do this, we'll divide the annual revenue by the expected number of inventory turnovers per year:
Annual revenue / Inventory turnover rate = Revenue per inventory turnover
$3,000,000 / 1.5 = $2,000,000
Now, we know that they plan to sell the cars at 4 times the purchase price. So, we need to find out the purchase price of the cars that will generate $2,000,000 in revenue per inventory turnover:
Revenue per inventory turnover / Markup factor = Purchase price of cars
$2,000,000 / 4 = $500,000
Therefore, Boll should expect its…
Reminds me of a blog post from a few days ago by Melanie Mitchell on a talk she gave about the recently-passed Brian Cantwell Smith:
Judgment, he wrote, is “a form of dispassionate deliberative thought, grounded in ethical commitment and responsible action, appropriate to the situation in which it is deployed.”
On the other hand, there is reckoning: the “calculative prowess” at which AI systems already excel. AI reckoning has lead to gold medals in mathematics competitions, to generating complex code, to predicting protein structure, and even to carrying out fluent conversations. But reckoning without judgment is a dangerous thing...
https://aiguide.substack.com/p/on-brian-cantwell-smith-and-the-promise
Careful. That claim about half of AI researchers was probably overestimating:
To summarize: The claim—that half of AI researchers believe that there is at least a 10% chance that rogue AI will kill us all—is based on a survey question that included 162 respondents. All we know about these respondents is that they were authors on papers published at 2021 machine learning conferences. Possible confounding issues are vagueness of the question, small sample size, response bias, confidence of responses, and level of expertise.
https://aiguide.substack.com/p/do-half-of-ai-researchers-believe
LLMS don't "hallucinate" they generate a stochastic sequence of plausible tokens that, in context when read by a human, are a false statement or nonsensical.
They also dont have an internal world model. Well I don't think so, but the debate is far from settled. "Experts" like the cofounders of various AI companies (whose livelihood depends on selling these things) seem to believe that. Others do not.
https://aiguide.substack.com/p/llms-and-world-models-part-1
https://yosefk.com/blog/llms-arent-world-models.html
Author's response: https://aiguide.substack.com/p/response-to-on-analogy-making-in