The UK AI Security Institute's blog publishes evaluations of frontier AI models, especially their cyber capabilities, and reports on AI trends.
Commenters use it to compare model cyber benchmarks, such as Mythos against GPT-5.5, and point to a follow-up post on how fast capability is advancing.
"This is just a wild exaggeration of what's happening here."
Is it? From the article...
"Both OpenAI and Anthropic said signing the agreement with the AI Safety Institute will move the needle on defining how the U.S. develops responsible AI rules."
From the UK AISI website with whom the data is also shared (their main headline in fact):
"Rigorous AI research to enable advanced AI governance" (https://www.aisi.gov.uk/)
The reality is that this would be a tremendous waste of time and money if all were just to sate some curiosity... which of course it isn't.
Let's look at what the US AISI (part of the Department of Commerce, a regulatory agency) has to say about itself:
" About the U.S. AI Safety Institute huuh
The U.S. AI Safety Institute, located within the Department of Commerce at the National Institute of Standards and Technology (NIST), was established following the Biden-Harris administration’s 2023 Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence to advance the science of AI safety and address the risks posed by advanced AI systems. It is tasked with developing the testing, evaluations and guidelines that will help accelerate safe AI innovation here in the United States and around the world." -- (https://www.nist.gov/news-events/news/2024/08/us-ai-safety-institute-signs-agreements-regarding-ai-safety-research)
So to understand what the US AISI is about, you need to look at that Executive…
The first part was an attempt at ironic humor, by repeating Gemini about the topic of offloading thinking to AI, not to be taken seriously as speculation or not.
As for the name changes..That is a fact you can look up, aswell as much analysis. It is my opinion that the move from framing this area from one of "saftey" to one of "national security", is interesting, and related to geopolitical movements towards "great-power", and ideological points of view that elevate "personal responsibility" and "reduced regulation" and is similar to long ongoing discussion in society like that about automobiles. I don't know if you call analysis speculation?
As for the part about Dimensionality. It is just my intuition — and so i suppose speculation — from some things like for instance the SolidGoldMagikarp glitch in early openai models.. How we understand all the way that there might be trigger certain outputs from a vastly large model? When those things can be completely opaque to human reason. Observability and Understandability are areas of research. I haven't seen anyone claiming that generative models outputs can be concretely controlled, thats why there is so many pre and post hoc work arounds.
So when a risk cant be eliminated, the question is how to manage it, and who's responsibility is…
There is little evidence that Mythos is any better at finding bugs than any other system. Mythos appears to be impactful because people are, for the first time, using lots of resources (for free from Anthropic) to try and find security issues. The actual bugs found are mostly inconsequential, any chart showing a giant leap in fixes that doesn’t consider whether they were even using any tooling before and whether these are serious issues is junk. If you read the partner’s summaries of Mythos so far, it is a damp squib. Maybe that’ll change but at least for now there is no evidence Mythos is anything but marketing hype.
https://www.aisi.gov.uk/blog/our-evaluation-of-claude-mythos-previews-cyber-capabilities
https://daniel.haxx.se/blog/2026/05/11/mythos-finds-a-curl-vulnerability/
AISI in the UK has been doing this for years - there are lots of papers https://www.aisi.gov.uk/category/safeguards and specific reports, e.g. this on GPT 5.5 https://www.aisi.gov.uk/blog/our-evaluation-of-openais-gpt-5-5-cyber-capabilities
This old post goes into lots of detail about what they do to red team and why: https://www.aisi.gov.uk/blog/early-lessons-from-evaluating-frontier-ai-systems
NIST's similar unit in the US is now called CAISI https://www.nist.gov/caisi - interesting that the most recent post is an evaluation of DeepSeek capabilities, which sound more like watching China. But presumably this executive order alters the emphasis?
> What changed with Mythos Preview is that a model can now take those low-severity bugs (which would traditionally sit invisible in a backlog) and chain them into a single, more severe exploit.
I think this statement seems to align with some of the other independent tests of Mythos[1]. It did very well on long agentic work which I expect is what they trained it for, and that requires being able to find these tangential links between loosely related topics in the context window.
[1] I'm mainly referring to https://www.aisi.gov.uk/blog/our-evaluation-of-claude-mythos-previews-cyber-capabilities
This is a really good report!
I was particularly fascinated about the self-replication section [0]; I wasn't aware of the RepliBench paper [1], and now that I skimmed through it, I'm quite concerned. It was clear to me before that there's nothing particular stopping an AI from replicating, but seeing the already high success percentage at current levels of technology (e.g. 46% success of replicating itself onto compute) is very scary.
[0] https://www.aisi.gov.uk/frontier-ai-trends-report#self-replication
[1] https://arxiv.org/abs/2504.18565