Joshua Rogers, a security researcher, writes about vulnerabilities, security tooling, AI-assisted bug finding, and network scanning.
HN commenters cite his review of AI SAST tools, which found many real bugs, and his comparison of ReDoS detection tools.
Okay, so anti AI people are just making shit up now. Got it.
According to Willy Tarreau[0] and Greg Kroah-Hartman[1], this trend has recently significantly reversed, at least form the reports they've been seeing on the Linux kernel. The creator of curl, Daniel Steinberg, before that broader transition, also found the reports generated by LLM-powered but more sophisticated vuln research tools useful[2] and the guy who actually ran those tools found "They have low false positive rates."[3]
Additionally, there was no mention in the talk by the guy who found the vuln discussed in the TFA of what the false positive rate was, or that he had to sift through the reports because it was mostly slop — or whether he was doing it out of courtesy. Additionally, he said he found only several hundred, iirc, not "thousands." All he said was:
"I have so many bugs in the Linux kernel that I can’t report because I haven’t validated them yet… I’m not going to send [the Linux kernel maintainers] potential slop, but this means I now have several hundred crashes that they haven’t seen because I haven’t had time to check them." (TFA)
He quite evidently didn't have to sift through thousands, or spend months, to find this one, either.
[0]: https://lwn.net/Articles/1065620/
[1]: https://www.theregister.com/2026/03/26/greg_kroahhartman_ai_kernel/
[2]: https://simonwillison.net/2025/Oct/2/curl/p
[3]: https://joshua.hu/llm-engineer-review-sast-security-ai-tools-pentesters
If something is found by Valgrind, we can reproduce it ourselves. Here we get private bug reports found by "his set of AI assisted tools".
The set seems to be:
https://joshua.hu/llm-engineer-review-sast-security-ai-tools-pentesters
So he likes ZeroPath. Does that get us any further? No, the regular subscription costs $200 and the free one-time version looks extremely limited and requires yet another login.
Also of course, all low hanging fruit that these tools detect will be found quickly in open source (provided that someone can afford a subscription), similar to the fact that oss-fuzz has diminishing returns.
I'm using them, but frankly only for things where my quality bar is lower. I've seen little evidence that anyone is routinely getting work that's comparable to competent humans, excepting incredibly recent work like Joshua Rodgers' [0]. And I'll note that even Joshua's work was mainly successful in identifying issues, but actually fixing them involved much more human intervention.
[0] https://joshua.hu/llm-engineer-review-sast-security-ai-tools-pentesters
Ultimately, one of the cameras in my iPhone 7 failed, and I replaced it with an SE3 - which was indistinguishable from the 7 - but another issue was patches.
With very little skill, I found I could duplicate this issue on my iPhone 7 running the latest supported version of iOS: https://joshua.hu/apple-ios-patched-unpatched-vulnerabilities
Wow very cool, theyve now closed 150 bugs identified via ai assistance/static analysis!
For ref, here is the post from Joshua Rogers about their investigation into the tooling landscape which yielded those findings
https://joshua.hu/llm-engineer-review-sast-security-ai-tools-pentesters
This should probably link to the original blog post by Joshua Rogers:
https://joshua.hu/llm-engineer-review-sast-security-ai-tools-pentesters ("Hacking with AI SASTs: An overview of 'AI Security Engineers' / 'LLM Security Scanners' for Penetration Testers and Security Teams")