Geoff Chappell

Extending the user shell was always for the benefit of Microsoft first, and for the rest of us second. Companies who tried to add serious advanced functionality through that found out that they were not really welcome, even though in theory everything was object-oriented, loosely coupled, language-agnostic, yada yada. It probably was mainly a solution to decrease the amount of territory wars over code inside Microsoft, and let teams work independently without demanding synchronous fixes.

We shouldn't forget that half of the documentation on how everything should work was only released after anti-trust investigation and pressure from courts.

https://www.geoffchappell.com/studies/windows/shell/index.htm

By that time, Windows had already switched to shiny fresh undocumented technologies.

You should remember that a lot of software around the Millennium simply re-implemented latest fashion trends (Office controls, XP styles, etc.) on their own in not quite exact ways, not just because programmers back then wore skins, and ate raw meat, and feared nothing, and because such third party toolkits were available commercially, but also because official interfaces to do the same were not offered until later releases (or sometimes ever). I vaguely recall discussions of teams within Microsoft sometimes doing the same, resulting in Office controls sharing no code with common controls used by the rest of the system which look exactly the same.

The irony is that web browsers, being the…

NT was designed to be portable, so the machine specific parts are cleanly separated, even on the same CPU. So if you port only those parts, the existing binaries for an architecture should work. The three bits are:

1) ARC boot firmware. NT was developed on non-x86 systems like i860 then MIPS, and ARC is the native boot firmware used (on x86 NTLDR emulated it prior to Vista). Similar to a PC BIOS / UEFI, and a PCI PowerMac would use OpenFirmware. As well as providing an ARC compatible environment that loads over OpenFirmware, this project seems to does some fun so the boot firmware can pretend there's a storage device so that driver "floppies" can be loaded during the initial stages of setup. (ie an F6 disc)

2) A HAL. The main NTOSKRNL is hardware agnostic, so the idea is that there's one binary for each CPU architecture. But the kernel needs to interface with actual timers, busses etc., so the interface code is in HAL.DLL and the appropriate one is copied by setup. (For example see https://www.geoffchappell.com/studies/windows/km/hal/history/index.htm for a list of x86-32 ones in older versions of windows, with various HALs for NEC PC-98, IBM MCA, various early multiprocessing systems, nowadays there's just one AMD64 one that's mostly in the kernel itself). So the main kernel in unaltered, and halgoss handles the Mac specific stuff.

3) Device drivers. Once NT is up and running it does need actual drivers.

The specs for 1) are known, so can presumably be emulated, and I…

> These days you need to have your kernel driver signed by Microsoft

However, this is required only for a “proper” kernel driver specifically; kernel code execution can still be accomplished without any signing at all using /dev/kmem-like mechanisms, which Microsoft explicitly does not consider a bug[1].

> or edit your boot config options to put the machine in an insecure state mostly useful for testing.

Or fiddle with undocumented registry settings (used, among other things, to support upgrades from Windows 7 installations with unsigned drivers) and suppress signing checks for your driver even outside of testing mode[2].

> To get it signed you need to pass a basic test suite which MS provides [...].

You also need to register a business entity and cough up upwards of 300 USD/yr for a Microsoft-approved EV code signing cert[3] before that, which is the biggest hurdle for me at least.

I have to say, even if this new Microsoft is not the same as old Microsoft, it sure looks very similar from some angles.

[1] https://github.com/ionescu007/r0ak#is-this-a-bugvulnerability-in-windows

[2] https://geoffchappell.com/notes/security/whqlsettings/index.htm

[3] https://docs.microsoft.com/en-us/windows-hardware/drivers/dashboard/get-a-code-signing-certificate

No offense, but I don't think you looked too hard. There are lots of calls that could be used to do things in a more performant way, or to make things possible that were not possible using documented APIs. [0]

Generally speaking though, what you say is true. There are certainly no "magically faster" API calls.

There are thousands of undocumented calls that, if you were aligned with the internal team at Microsoft that wrote them, allowed you to write better performing code than if you were a third party. And as a major developer inside Microsoft, you had the power to get API calls created for your needs, while third parties didn't. That is the crux of the matter.

Here's one straight from the site we're discussing today, the IOwnerDataCallback interface [0] makes it possible to do grouping on a virtual ListView. Windows Explorer uses it to great effect, everyone else can't use it. [2]

[0] http://undocumented.ntinternals.net/

[1] https://www.geoffchappell.com/studies/windows/shell/comctl32/controls/listview/interfaces/iownerdatacallback.htm

[2] https://www.codeproject.com/Articles/35197/Undocumented-List-View-Features

The only other equivalent API in broad use today that I'm aware of would be the Kernel32.dll subset of Win32 [1]. Not that it's perfect (so much accumulated cruft) or what I would call modern (still has a lot of ambient authority baked-in) by any means, but the important parts are mostly handle-based (unlike PID-based POSIX process management functions for example).

I happen to like the design of the syscall layer of Fuchsia's Zircon kernel [2], but it's not a full substitute for POSIX (notably, file and network I/O are built as userspace features of top of IPC channels, they are not kernel concepts).

[1] https://www.geoffchappell.com/studies/windows/win32/kernel32/api/index.htm

[2] https://fuchsia.dev/fuchsia-src/reference/syscalls

This brought to mind the AARD "crash" which Microsoft used to basically destroy competition from DR-DOS back in the day.

> The AARD code was a segment of code in a beta release of Microsoft Windows 3.1 that would determine whether Windows was running on MS-DOS or PC DOS, rather than a competing workalike such as DR-DOS, and would result in a cryptic error message in the latter case. This XOR-encrypted, self-modifying, and deliberately obfuscated machine code used a variety of undocumented DOS structures and functions to perform its work.

https://en.wikipedia.org/wiki/AARD_code

https://www.geoffchappell.com/notes/windows/archive/aard/drdos/index.htm

https://news.ycombinator.com/item?id=36042213