Re: [Ntp] Antwort: Re: The bump, or why NTP v5 must specify impulse response

[Warner Losh <[e-mail]>]

On Fri, Apr 17, 2020 at 8:53 AM Daniel Franke <[e-mail]> wrote:

> On Fri, Apr 17, 2020 at 6:21 AM <[e-mail]> wrote:
> >
> > Hey all,
> >
> > I wanted to offer my point of view on the question of modularity for
> NTPv5 as discussed here.
> > As many of you may know, the place that I'm coming from is not decades
> of practical experience in NTP implementation and/or maintenance, but
> instead years of close academic and theoretic scrutiny concerning
> guarantees for time synchronization methods.
> >
> > For me personally, the separation of NTP would ideally not only be made
> into protocol on the one hand and algorithmic and response behavior on the
> other hand, but go one step further.
> > I would prefer separation into three layers, specifically:
> >
> > 1) Network protocol (what kinds of messages can one send, and how does
> one have to respond to incoming messages); this should be specified on a
> per-association basis and security should be handled mostly here.
> > 2) Algorithmic offset calculations to the desired "true" time scale (how
> does one calculate time offset, frequency offset, perhaps offset of
> higher-order derivatives); this can involve multiple associations; security
> of the individual associations should be known so it can be taken into
> account if desired; this is the module that needs to be aware of the use of
> different time scales such as TAI vs. UTC, the handling of leap seconds
> etc.; this module should also be aware of how clock steering is handled
> (see below) and account for it.
> > 3) Clock steering (how is the participant's clock manipulated in order
> to accomodate the measured offset(s) and make the clock read something
> close to the "true" time, e.g. hard-setting the counter vs. manipulating
> the frequency via counter increment vs. manipulating the clock in other
> ways such as via the oscillator's voltage vs. handling all of that via
> virtual clocks); this can differ between machines, specifically between OS
> and hardware and should be treated appropriately.
> >
> > With this separation, we need well-defined interfaces between 1) and 2)
> and between 2) and 3).
> > Not dealing with 3) in a specification and leaving it to implementations
> should be seen as a real option, keeping in mind that knowledge of 3) can
> increase accuracy of 2).
> >
> > Regarding the nomenclature, I would strongly suggest getting consensus
> first on which of the many ways forward is going to be taken in order to
> arrive at something that can be called "NTPv5".
> > Specifically, it should be decided early which of 1), 2) and 3) that
> document should treat, the options as I see them being 1) only, or 1) and
> 2), or all three.
> >
> > What do you all think?
>
> My ontology is similar, but quite identical, to Kristof's:
>
> I. The syntax and semantics of the wire protocol
> IIa. The algorithm for estimating the offset between your clock and
> that of your peers
> IIb. The algorithm for combining estimates collected from multiple
> peers into one "consensus" estimate
> IIIa. How to use that estimate to steer the clock used for capturing
> the timestamps used in the wire protocol
> IIIb. How to use that estimate to steer the clock that other
> applications see when they ask for the system time
>
> The central question of this thread so far, up until Kristof entered,
> could be expressed as "How much is (IIIa), and by extension (IIa) and
> (IIb), really a part of (I)? When we say that the timestamps on the
> wire confer an estimate of the current time, how particular do we need
> to be about the properties of that estimate?".
>
> NTPv4 and prior don't distinguish at all between (IIIa) and (IIIb),
> but NTPv5 might have to. My sketch allows discontinuous adjustments to
> the `tai_loc_offset` field with each new data point obtained. But a
> lot of applications depend on having a single system clock that's all
> at once monotonic, reasonably accurate, and reasonably
> frequency-stable, which means there has to be some further smoothing
> step involved (perhaps a PLL, perhaps something else).
>
> In an ideal world, the contract between the OS and the NTP daemon
> would be the same on all platforms, and would be something like this:
> the OS provides the NTP daemon with a monotonic time counter driven by
> the hardware oscillator, and the NTP daemon provides the OS with a
> running estimate (maybe with error bars included) of the first few
> Taylor series coefficients of the function from that counter to TAI.
> What the OS does with this information and in particular how it
> communicates it to other applications on the system is none of the NTP
> daemon's concern.
>

Counters of sufficient resolution wrap. There's often a software counter
too. And getting either the raw software or raw hardware counters and/or
their notion of a relationship to the local system time can be tricky as
well. Some OSes allow access to the raw counters (since CPUs allow turning
that access on/off), and some even publish what's basically a 'phase at
count' testimate and a frequency estimate that the applications can then
use to compute time w/o a round trip into the kernel (this is often hidden
inside of libc's clock_gettime or similar).

Also, how does NTPd get access to this pure counter when it timestamps the
packets? It would need to know that in order to measure the evolution of
that counter over time to make estimates about it.

And I'd maintain that the NTP daemon needs to provide both the phase
estimate of the counter vs the TAI and a shorter-term frequency estimate
for the counter's evolution. Since the counter is likely to be lousy over
time and the frequency will wander all over the place (relative to a high
precision time keeping xtal or clock), the most useful information is that
counter X is TAI time Y and the short-term frequency is Z (maybe with a Z'
estimate as well). The OS will need to read the counter and produce a
TAI/UTC time quite often, and doing more than a simple computation will
show up in benchmarks. But that thinking may be a legacy of NTP steering
the OS's notion of time that it gets from a pure counter under the covers.
This is why we have ntp_adjtime providing a time delta and frequency error
and a bunch of other stuff that talk about ntpd's notion of how good those
are which are on a spectrum from generally useful, to totally NTPd
specific. I have a hard time seeing a way to not push the error in time /
frequency measurements down.

Second, the OS needs to keep a smooth time to its clients w/o time jumps
(but small changes in frequency are OK). So there will likely need to be
some coordination between the OS and the ntp daemon to communicate how much
of the 'steers' or 'tyler series' information it could ingest and what it's
notion of a future counter's time will be so NTP can help steer the OS's
use of the counter so it will be on time in the future and not overshoot as
the frequency wanders around.

Unfortunately, currently *no* OS provides us with anything this clean,
> and even if one did we'd still have to accommodate all the ones that
> don't if we want our spec to be platform-agnostic. Since we don't
> control the OS landscape and it changes on a different and faster
> schedule than NTP spec does, there's only so much we can do here. We
> can specify what we wish the interface would look like and maybe some
> kernel hackers will feel inspired by it, but for the foreseeable
> future NTP implementers are going to have to contend with
> less-than-ideal interfaces, and the most the IETF can do for them is
> give some abstract general guidance on that.
>

I think this sounds great from a theoretical view. I think it sounds lousy
from a practical view.

Generally, timekeeping is a critical part of the OS. This makes the above
ideal not implementable in many ways.

CPU Cost: If the OS is having to do something complicated to convert the
internal counters to a time as often as it needs time, the interface will
be a non-starter.

DoS issues: What does the OS do when ntpd information goes away? Because it
will. And then it will come back and the OS will have evolved its time
based on hardware counters whose frequency estimates will have been used as
best they can, but they will likely be wrong.

Convergence: How do we know NTPd's recommendations will converge with
whatever is going on inside the OS? Is there a stable form or do we get
oscillations. This is the same issue, btw, with algorithmic improvements
talked about elsewhere: you need to make sure that whatever is going on
with the steering of time doesn't cause oscillations as you resynchronize
with your upstreams. This will be especially an issue with pools of NTPd
servers that are steering using different algorithms. Steering to a steered
clock is tricky and well known problem in the high-precision time-keeping
biz. Often times I created a measurement of a free-running clock and
published my estimates of that clock so that people making measurements
could make adjustments after the fact since they needed a high precision
measurement of events, but didn't need it at the instant the measurement
was made. Had we steered the clock, there'd be lags to that accuracy. And
if they were steering their clocks, they'd get oscillations.

Warner