Dougall Johnson writes low-level technical posts on Apple M1 performance, Rosetta 2, SIMD, and bit-twiddling optimizations.
Commenters cite his Rosetta 2 post for how Apple CPUs speed up x86 emulation, and find his bit-twiddling posts helpful.
> (even if they can actually execute all instructions concurrently, which is not clear to me and unlikely - Intel can also only execute some instructions on certain ports)
It's pretty symmetrical, moreso than Cortex-X2's 4 pipelines; there's analysis that on M1 only some floating point and crypto instructions can't execute on all 4 pipelines. [1] (TP in that table is inverse throughput)
Which means that, for example, byte permutes from tables 256b or less can actually achieve the same throughput on M1 as with Intel's AVX-512, since M1 can sustain 4x 2-register TBL per cycle. And doing the exact equivalent of a 512b vpermb (3 cycle latency, 1/cycle throughput) can be done with 5 cycles latency and 0.33/cycle throughput on M1, via 4x 4-register TBL.
Well, a vpermd in NEON would need an extra MLA to convert indexes, and vpermi2* equivalents fall off a cliff. And Intel still has p01 free, and COMPACT is SVE. But in general, a lot of the parallelism that enables AVX-512 implementations will convert directly into ILP across 128b vectors.
> This seems unlikely because we're sorting 8 MB and my understanding is that cores (unless L2==LLC) generally have private, partitioned L2 caches, so 3 MB in the case of M1. Is that incorrect?
Anandtech [2] measured the same L2 latency up to about 8MB single-core, so regardless of the details, 8MB is a pretty significant cliff on M1. Regardless, RAM bandwidth is ~60GB/s, and unlike Intel, can be just about saturated by a single core.
[1]…
Oh, I see what you are saying. I don't consider it to be a heuristic because it's simply bruteforcing it. IMO a heuristic needs improve over brute force.
> And 4 vs 8 is a pretty sizeable difference.
True, but x86 was doing four instruction 20 years ago. As I mentioned the current state of the art (in a shipping product) is 9, and 9 is larger than 8. Importantly, this Skymont of leapfrogging decoders approach is scalable.
> whereas Apple can just decode each op independently & only decodes 8.
Apple isn't as free from serialisation as you suggest. Like X86, many instructions decode to multiple uops. According to research [1] instructions which decode to two uops are common and a few decode to as many as 12 uops.
It also does instruction fusion, two neighbouring instructions can sometimes decode into a single uop. This all means that there is plenty of serialisation within Apple's decoder. And branching also creates serialisation.
It's just not as simple as independently decoding eight instructions into eight uops every cycle. Simpler than what x86 implementations need to do, but not as brain-dead simple as you suggest.
Actually, Skymont's approach has an advantage over Apple here, because it only needs to serialise within each 3-wide decoder.
[1] https://dougallj.github.io/applecpu/firestorm.html
> I believe Rosetta 2 also has a JIT mode
It does have JIT translation (not a JIT "mode" though, as it always use AOT translation, only relying on JIT translation at runtime for the parts that need it)
> which is a bit more like conventional emulators
Not at all†, Rosetta 2 does the same†† translation step on dynamic Intel code, whose arm64 output can be reused afterwards
> But it's used infrequently, eg when dealing with x86_64 apps that themselves use a JIT
Yes, although it's more like "exceedingly rarely" in practice since usually those interpreters are up to date enough to have a native arm64 release.
See there for details: https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-fast/
† Unless you've been meaning dynarec, but I would not call that "conventional" although it is a well-known technique https://en.wikipedia.org/wiki/Dynamic_recompilation
†† IIUC minus a few things that can't be done when just-in-time because some assumptions are not guaranteed to be satisfied.
If I'm reading this analysis [1] correctly, M1 has only one integer divider, but it appears to be almost fully pipelined as it can indeed start a new divide every other clock cycle.
The latency is also impressively good even at 64 bits, but the benchmark should not be div-latency bound in any case as, each division can be started without waiting for the previous division to finish.
edit: of course it is possible that internally the divider is not fully pipelined, but there are actually multiple dividers. It is only exposed as a single unit (i.e only a single port) because it wouldn't be able to start more than one operation per clock cycle in any case. If I had to guess, the additional SIMD dividers are repurposed for this, but that doesn't explain why 64 bit still has the same throughput as 32bit.
edit2: SIMD is on the same port as fdiv, not integer. fdiv has a throughput of 1 div per clock cycle!
[1] https://dougallj.github.io/applecpu/measurements/firestorm/SDIV_fast_32.html
> There are only a handful of different instructions that account for 90% of all operations executed, and, near the top of that list are addition and subtraction. On ARM these can optionally set the four-bit NZCV register, whereas on x86 these always set six flag bits: CF, ZF, SF and OF (which correspond well-enough to NZCV), as well as PF (the parity flag) and AF (the adjust flag).
> Emulating the last two in software is possible (and seems to be supported by Rosetta 2 for Linux), but can be rather expensive. Most software won’t notice if you get these wrong, but some software will. The Apple M1 has an undocumented extension that, when enabled, ensures instructions like ADDS, SUBS and CMP compute PF and AF and store them as bits 26 and 27 of NZCV respectively, providing accurate emulation with no performance penalty.
https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-fast/
The flags get saved and restored on task switches and interrupts. I would think deciding whether those bits actually get used across that is infeasible, but yet, https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-fast/ says:
“This almost entirely prevents inter-instruction optimisations. There are two known exceptions. The first is an “unused-flags” optimisation, which avoids calculating x86 flags value if they are not used before being overwritten on every path from a flag-setting instruction.”
Does this mean the emulator is buggy (if I write a loop that sets and clears flags, but doesn’t do anything with them, an interrupt handler that samples the flag values would need to see both values), but in a way that no sane code would notice?