Naked Security

It is possible to create customised UNIX-like OS with custom kernels, read-only filesystems (mounted images) and writable directories mounted as tmpfs, booting from removable USB. Some call this "diskless". There are some drawbacks from using USB media such as lack of a good randomness source on boot but there are many advantages. Kernels and filesystems are just single files on the USB media and can be easily switched/updated. The system stays "clean". It is "like new" on every reboot. I was doing this with a netbook long before the so-called "Chromebook" came along.^1

The problem with Chromebooks is that they are designed to try to get people to "log in" to Google and to use "the [Google] cloud" for storage. Chromebooks in Guest Mode have an array of Google-authored daemons running the the background from read-only media. There is no way to disable them. You cannot even change the options passed to Chrome, e.g., to disable "Origin Trials". This setup is great if you love everything Google, but not great if you just like computers, you bought the computer for the hardware and drivers, and prefer to choose your own software. With these Google programs always running in the background, it means you do not have ultimate control over the computer, Google does. Another annoying thing is that ChromeOS, as well as Chrome, is a WIP. It is constantly changing. For example, bluetooth may be working fine and then suddenly there is an "automatic update" that breaks it. …

I'm a little confused with your response, or we appear to be talking past each other.

For context, I'm a former pentester (NCC Group, formerly Matasano). I've been an ML researcher for four years now, so it's possible I have a unique perspective on this; the combination of pentester + ML is probably rare enough that few others have it.

> You cannot inject a valid-looking system message from user text.

https://greshake.github.io/ did exactly that. (HN discussion: https://news.ycombinator.com/item?id=34976886)

Take a look at this screenshot: https://greshake.github.io/resources/demo.png

Now, I understand it's possible that Bing was using an older version of your ChatML format, or that they did something dumb like inserting website data into their system prompt.

But you need to anticipate that users will do dumb things, and I strongly recommend that you prepare them with some basic security recommendations.

If the Bing team can screw it up, what chance does the average company have?

I suspect what happened is that they insert website data into the system text, to give Bing context about websites. But that means that the attack wasn't coming from user text -- it was coming from system text.

I.e. the system text itself tricked system to talk like a pirate.

This is known as a double-escaping problem in the pentesting world, and it pops up quite a lot. In this case, an attacker was able to break out of the sandbox by inserting user-supplied text (website data) into an…

The "goto fail" bug isn't a great example; Rust actually does specifically guard against this by requiring curly braces around the body of "if".

Here's an explanation of "goto fail", which includes a code snippet.

https://nakedsecurity.sophos.com/2014/02/24/anatomy-of-a-goto-fail-apples-ssl-bug-explained-plus-an-unofficial-patch/

If C conditionals required explicit braces around the body, the second fail would have been redundant code instead of a security vulnerability.

Additonally, Rust will warn you if you have "unreachable" code, such as all the code after the unconditional "goto fail". Though, I'm sure modern C compilers would also warn about this.

All that said, Rust definitely does have plenty of bug classes left for people to trip over. Integer under/overflows come to mind (though they're just logic errors instead of undefined behavior).

Not disputing what you say, but mistakes do happen to the best of us.

http://sunnyday.mit.edu/accidents/Ariane5accidentreport.html
and yet again another OpenSSL bug , another flaw in the open source fallacy of many eyes to spot bugs...
https://nakedsecurity.sophos.com/2021/03/28/serious-security-openssl-fixes-two-high-severity-crypto-bugs/

plus when considering in maths there are ways to rearrange formulas including new discoveries like this one (a new way to do quadratic equations)
https://news.ycombinator.com/item?id=29342161
https://www.sciencealert.com/math-genius-finally-discovers-easy-way-to-solve-quadratic-equations-after-4-000-years

I wonder if discovering a new way to do things could be done to cryptographic algorithms.

Link in the article comments to http://nakedsecurity.sophos.com/2011/09/21/microsoft-dumps-partner-telephone-support-scam/ is worth reading. Amazing that someone would have the balls to try this, but I suppose there are no penalties for what amounts to abuse of privileges. I know MS tightened up on some aspects of the partner scheme (presumably to stop everyone and his dog using it as a way to get free copies of Windows).

So, who owns Comantra? There needs to be an international register of people unfit to own companies, or whose companies one shouldn't do business with. In the UK, Companies House records the really bad guys who've managed to get struck off as directors, but this kind of scam needs to get you on a little list too.

It's weird, and even harmful in many cases.

A modern goto of sorts: it has performance benefits [1], but not overall readability ones.

switch (x) {
case 0:
doThing1();
break;
case 1:
doThing2();
break;
default
doThing3();
}

vs

if (x == 0) {
doThing1();
} else if (x == 1) {
doThing2();
} else {
doThing3();
}

This somewhat unsafe form of switch-case made it into C, Java, C#, and JavaScript, and cases the great iOS SSL bug of 2014. [2]

[1] https://en.wikipedia.org/wiki/Duff%27s_device

[2] https://nakedsecurity.sophos.com/2014/02/24/anatomy-of-a-goto-fail-apples-ssl-bug-explained-plus-an-unofficial-patch/