David Mills

[author]

> I wish he'd explained somewhere how they leapt to examining virtualized NTP clients...

I had a hunch [wrongly] that the traffic was caused by a particular operating system. I didn't have enough machines to run the tests on bare-metal, so I virtualized them. And I suspected that virtualization would provide a worst-case scenario (the virtualized clocks would be jittery).

My big surprise was that Windows was a model client (once per day), OS X was good, too, and that FreeBSD was the worst and Ubuntu a close second. It was a complete inversion of what I had expected to find.

> what they ultimately did (since there's no part 3 that I can find).

I never wrote part 3, but now that it seems the post has gained traction on Hacker News I might be inspired to write one. The summary would be as follows:

To reduce your costs by a quarter, use the following lines in your NTP configuration file to throttle overly-aggressive clients (the important directive is `limited`:

```
restrict default limited kod nomodify notrap nopeer
discard minimum 0
```

Here is a description of NTP rate-limiting and why `limited`, `kod`, and `discard` are important: https://www.eecis.udel.edu/~mills/ntp/html/rate.html

Here is my current NTP configuration:
https://github.com/cunnie/deployments/blob/95e9c71e882d453ec0fd168f3a8030ea0707c05a/concourse-ntp-pdns-gce.yml#L178-L192

I'm a bit rusty, but I believe the way NTP works (at least the reference version which is commonly used) is that if a client sends too many requests in a short time, they are ignored except to reply with a "back off packet" which is called the KoD (Kiss of death) in NTP terms.

Security audits have found some issues with abusing the KoD so I'm not sure if it still works like that or if it tends to be disabled. (I was on one of the teams doing the audit, I found the "Skeleton Key" defect)

https://www.eecis.udel.edu/~mills/ntp/html/rate.html#kiss

If you wanted to help the server deal with DoS even better, I would guess the best solution is to put a rate limiting firewall in front of it.

A good portion of the size difference (though certainly not all) is that ntp.org's ntpd has a lot of drivers to interface with physical devices [1], while OpenNTPD doesn't. If you don't own the physical devices, the LoC in the drivers are kind of a wash: they're not useful to you, but you also won't be running them. A stripped-down NTP-client-only build option would be nicer, though. PHK's suite of ntpd replacement tools seems to be going in that direction, offering a client-only tool for edge servers that just need to sync time over the network.

[1] http://www.eecis.udel.edu/~mills/ntp/html/refclock.html

An "fun" use of ip addresses is in NTP.

in the ntp config file, you will have stuff like this:

server 127.127.1.0 # local clock

or:

server 127.127.20.0 minpoll 4 iburst prefer # gps clock

where the "ip address" is of the form: 127.127.<clocktype>.<instance>

here's a page explaining the clock types:

https://www.eecis.udel.edu/~mills/ntp/html/refclock.html

but basically it's a weird anachronism. I'm not sure if NTP will actually bind to those addresses using the tcp/ip stack, or if it someone just got lazy and coopted the ip address parser for off-label use.

Well, quoting from: http://www.eecis.udel.edu/~mills/ntp.html

"Used in the Internet of today with computers ranging from personal workstations to supercomputers, NTP provides accuracies generally in the range of a millisecond in LANs and up to a few tens of milliseconds in the global Internet"

But this is news to me, interesting:

"When kernel support for precision timing signals, such as a pulse-per-second (PPS) signal, is available the accuracy can be improved ultimately to the order of one nanosecond in time and one nanosecond per second in frequency."

Of the various internet .+P, NTP is one I never learned about as a student, so now I'm looking at its web page [1] by its creator David L. Mills (1938-2024). I've found one video of him giving a retrospective of his extensive internet work; he talks about NTP at 34:51 [2] and later at 56:26 [3].

[1] https://www.eecis.udel.edu/~mills/ntp.html

[2] https://youtu.be/08jBmCvxkv4?si=WXJCV_v0qlZQK3m4&t=2092

[3] https://youtu.be/08jBmCvxkv4?si=K80ThtYZWcOAxUga&t=3386