Easyperf

Replacing hardware memory safety with WASM is a great idea, but FWIW you can get most of the benefits by just using hugepages. Eg, clang spends 7% of its time on TLB misses and you can get a free 5% speedup using hugectl: https://easyperf.net/blog/2022/09/01/Utilizing-Huge-Pages-For-Code

The practical solution would be to replace the MMU with a tiny one that's backwards compatible and extremely cheap in silicon area and latency. Old binaries would limp along due to translation overhead while new software would use only a couple of pages across the whole system since memory protection is done in software when the bytecode is compiled and you never need to swap to disk. New binaries would run >>7% faster. I wouldn't call this a high-level CPU tho. Most high-level CPU ideas are really stupid ones like tagging every word with a type ID that must be checked by hardware while wasting cache on something the compiler could have done.

> If you get rid of branch prediction and prefetching logic you get more space for more cores. If you hook up memory in parallel to those cores you get higher effective memory throughput and executed instructions per second.

Sounds like a GPU. Compiling normal code to run on chips like that is still an open problem and I doubt it will be solved within the next decade. We'd need AGI to rewrite code the same way humans do

> CPUs already contain trie walkers for their page tables, you might as well use that for looking up data in indexes directly.

I…

I've looked into this before and there are very few tools for this. The only vaguely generic one I've found is Codespeed: https://github.com/tobami/codespeed

However it's not very good. Seems like most people just write their own custom performance monitoring tooling.

As for how you actually run it, you can get fairly low noise runtimes by running on a dedicated machine on Linux. You have to do some tricks like pinning your program to dedicated CPU cores and making sure nothing else can run on them. You can get under 1% variance that way, but in general I found you can't really get low enough variance on wall time to be useful in most cases, so instruction count is a better metric.

I think you could do better than instruction count though but it would be a research project - take all the low noise performance metrics you can measure (instruction count, branch misses etc), measure a load of wall times for different programs and different systems (core count, RAM size etc.). Feed it into some kind of ML system and that should give you a decent model to get a low noise wall time estimate.

Good tips here:

https://llvm.org/docs/Benchmarking.html

https://easyperf.net/blog/2019/08/02/Perf-measurement-environment-on-Linux

Clang/LLVM accepts (little-known but documented) flags to let you align code block using a custom alignment, though it only works at file-level. See [1].

However, I'm not sure if doing so is useful or necessary. Interpreter performance is sensitive to code layout (which affects hardware branch predictor accuracy), but I don't think there is a general way to optimize the code layout to make the branch predictor as happy as possible.

So if you changed your code alignment and saw a perf change, it's more likely caused by the random perf gain/loss due to code layout change, not because that 1-byte-alignment is better than 16-byte-alignment or vice versa.

[1] https://easyperf.net/blog/2018/01/25/Code_alignment_options_in_llvm

I can definitely recommend https://book.easyperf.net/perf_book

The author's blog has been consistently great throughout the years, https://easyperf.net/notes/

See also microarchitectural performance analysis tools & readings, https://github.com/MattPD/cpplinks/blob/master/performance.tools.md#microarchitecture and "Comments on timing short code sections on Intel processors", http://sites.utexas.edu/jdm4372/2018/07/23/comments-on-timing-short-code-sections-on-intel-processors/