Entropic Thoughts is a personal blog on programming, Haskell, risk analysis, decision-making, and LLM benchmarks.
HN commenters call its posts a fantastic read, citing its STPA analysis of an AWS outage and its pieces on nomograms and Haskell.
> we've had say superhuman competitive programming performance for awhile
Fair. Question though, is this when compared to competitive programmers, or developers in general?
> extremely strong performance (super-p90-engineer) on say language-to-language porting
I'd need to see the methodology here and could easily be wrong, but I suspect this is largely down to "faster" and "willing to do a lot more of it without complaining"
> RE-bench (ML research engineering benchmark from METR) is already clearly above human perf
This pretty much has to be "relative to devs who don't specialize in that area", because if it wasn't the frontier labs wouldn't be paying a fortune to hire ML researchers.
> Mythos clearly has superior cyber capabilities
Based on Daniel Stenberg's experience with it [0], it seems like it's at best roughly on par with human experts. It's advantage is cost/speed.
> Also, why do you discount speed and cost so much?
Because in all the domains LLMs are applicable to, getting something cheaper/faster at the expense of quality isn't new or particularly interesting.
> Every model iteration is a significant bump up in performance according to a lot of complementary and principled measurements. What's been the thing that hasn't been true?
That they were good enough. To reuse the baby analogy, if every week your friend told you that their infant child was now heavier than an elephant (while acknowledging that the baby was lighter than one the previous week),…
Insurance can actually be a win-win if you keep your money sitting anywhere else other than a literal piggy bank.
Basically, it takes just as much time for your investments to go from 10,000 USD to 50,000 USD as it does for 50,000 USD to go to 250,000 USD. So a setback of 5,000 USD has a disproportionate impact on your future financials the less money you start off with.
In other words, if your total wealth is low enough, the premiums can set you back much less than the expected loss from the insured event, and at the same time make insurance companies a profit.
I'm still learning about this stuff, but here's an article that breaks it down in more detail and more clearly than I have: https://entropicthoughts.com/when-is-insurance-worth-it
Said article has apparently popped up on HN a few times: https://news.ycombinator.com/item?id=26834333
Yeah, but what is worst? I tried to understand the Haskell way in an article last week[0]
pure (n, guard (factor /= n) $> factor) Which I think is more or less the same as this python line.
return [factor for factor in factors if factor and not factor == n] The article does fancy stuff with memory caches which I believe is easy to do in python but I need to understand the Haskell code better.
[0] Haskell: A Great Procedural Language https://entropicthoughts.com/haskell-procedural-programming#leaning-into-the-first-classiness-of-effects https://news.ycombinator.com/item?id=42754098
So far my children have not yet had to repeatedly perform complicated calculations, but I look forward to the day. I will definitely teach them with nomograms before we go on to spreadsheets!
Another type of almost-nomogram that's great and practical is the slide rule. In particular in the kitchen, where it makes it really easy to translate proportions. https://entropicthoughts.com/kitchen-slide-rule
Statistical control is a very specific term: https://entropicthoughts.com/statistical-process-control-a-practitioners-guide, and one which you'd expect anyone with a significant interest in Deming to understand.
My expectation is that Lorin would read the parent comment and say some variant of "oh, whoops, I didn't check." As the parent noted, it's not really that important to the overall point.
> This is nonsense. A standard parser takes one input and does processing of this data to give an expected output.
This appears to be the root of your confusion.
Here's a good example of a parser: https://entropicthoughts.com/parser-combinators-parsing-for-haskell-beginners
There are at least 16 paths through this function.
Not one.
16.