Pavel Zemtsov

> I like the idea of software, libraries and application code all being accessible to each other and have the ability to call each other. Too much functionality is siloed behind application A or B, and the interface exposed by these applications is often woefully inadequate for anything interesting.

Maybe I'm skeptical, but I've gotten to the point where I believe that this specific problem is not going to be solved anytime soon.

Here's how I see it. I believe that unification, specifically within software, is universally based on the idea that we can clean up the world (where "the world" refers to a given software environment) if only we can reimagine it clearly enough and, to borrow a few technical concepts, compute a delta with enough detail and resolution that it'll carry us from where we are to where we want to be. (With the implicit [mis]understanding that all we need to do is make said delta exist, and then it'll do all the work of carrying us.)

I think the above is what we're trying to do - and that we're doing it because we're staring at/barking up the wrong set of trees (the associated forests of which got lost long ago).

Ultimately, the world is a disorganized mess. Any security researcher could entertain you for days with highly-detailed hair-raising examples and proofs of this. I've very recently (finally) realized this is because, at a very high level, the world operates somewhat similarly to the bazaar model Linux uses. Evolution is highly organic, often…

> Optimizers don't look for undefined behaviour to then do whatever, they look for the set of defined behaviours and then try to generate the cheapest code that behaves correctly in all those cases--and if that means that undefined cases do weird stuff, that's an acceptable side effect.

> if any of the bytes are not inside any valid C object, then that would produce undefined behaviour

The compiler might also assume that every one of these loads will be exactly 4 bytes apart. Even worse, it might know that int32_t allocations will always be aligned, so it will assume that every load will be 4-byte-aligned. It then might output code that loses track of the bottom two bits, or SSE instructions that only work with certain alignments.

This isn't theoretical, this exact kind of bug has happened before, on a normal non-malicious compiler: http://pzemtsov.github.io/2016/11/06/bug-story-alignment-on-x86.html https://news.ycombinator.com/item?id=12889855

It's true that the buffer overrun issue is the same for either version; I misread what was being replaced. But the alignment issue is specific to this version, and can't be ignored.

Oh I see you didn't get to caveat 5: you can't read anything more complicated than raw bytes, i.e. chars, because of unaligned memory access errors. Let's say you mmap a file and do something like this:

char *fileContents=...mmap etc...;
int headerOffset=*(int*)fileContents;
int *someListOfNumbers=(int*)(fileContents+headerOffset);
int importantSum=
someListOfNumbers[0] +
someListOfNumbers[1] +
someListOfNumbers[2] +
someListOfNumbers[3];

If headerOffset is not a multiple of 4, bad things will happen. On x86 you'll get away with it... Unless you were summing many more ints and the compiler decides to use aligned SSE loads for speed[1]. On ARM you'll get a SIGBUS just for trying to read unaligned ints (IIRC). You can fix that by wrapping your ints or whatever you're trying to read in a packed struct, but it is yet another thing to keep in mind.

[1] http://pzemtsov.github.io/2016/11/06/bug-story-alignment-on-x86.html