Re: [Ntp] WGLC: draft-ietf-ntp-interleaved-modes

[Harlan Stenn <[e-mail]>]

Miroslav,

I'm happy to work with you on this, if you're interested and think it
would be useful.

I note:

- 18ns corresponds to a precision of about -26.  While that's pretty
special, I remain curious about how long the delay is between pulling
the transmit stamp and the packet going out over the wire.  If the basic
precision of the system clock is only -19, we're not going to do better
than 2 microseconds.  For situations like this, I'm really curious if
we'll even see any benefit to client/server interleave mode.

- I'd like to explore a "follow up" message as an alternative.

- I'm wondering if we can do all of this with "plain" packets, or if
there is benefit to using an extension field for some of these exchanges.

H

On 12/11/18 4:48 AM, Miroslav Lichvar wrote:
> On Mon, Dec 10, 2018 at 05:47:27PM -0800, Harlan Stenn wrote:
>> On 12/10/18 8:05 AM, Miroslav Lichvar wrote:
>>> The document says "The server SHOULD discard old timestamps to limit
>>> the amount of memory needed to support clients using the interleaved
>>> mode." and "The server MAY always respond in the basic mode."
>>
>> I'm not talking about old timestamps.
>>
>> I'm talking about a case where there are LOTS of current clients.
> 
> Maybe I'm not using the word "old" correctly? If there is a large
> number of clients trying to use the interleaved mode, the server needs
> to keep a large number of timestamps to be able to respond in the
> interleaved mode to all of them.
> 
>> The client/server portion of this proposal changes the daemon from
>> stateless to stateful.
> 
> Yes. A state is needed to save the transmit timestamp. There is no way
> around that. If it was sent immediately in a separate message, it
> would amplify the traffic, or the length of the response wouldn't be
> symmetric to the request.
> 
> It's a bit like a TCP-based service. If there is a large number
> of clients, or there are DoS attacks, there is no guarantee that a
> client will get an interleaved response.
> 
> An important difference to TCP is that the protocol gracefully falls
> back to the basic mode. Clients can still synchronize to the server.
> They must not expect the server to be able to respond in the
> interleaved response.
> 
>>> There is a minimal client and server implementation in python. Please
>>> feel free to test it and report if anything breaks.
>>>
>>> https://github.com/mlichvar/draft-ntp-interleaved-modes
>>
>> You have this in chronyd then? 
> 
> No, in chrony it's implemented differently. The server support was
> basically bolted on the monitoring facility. That was the easiest and
> safest way. In ntpd it might be the same.
> 
> There is only one timestamp per IP address, which means it doesn't
> work over NAT (as documented in the man page). Some of the code is
> shared with the symmetric mode and there are optimizations, so it may
> be difficult to see what is actually going on in the client/server
> interleaved mode.
> 
> For implementors interested in the client/server interleaved mode, I'd
> rather recommend to first look at the python example.
> 
>> Do you have any results on how much of a difference it makes?
> 
> It corresponds to the difference in accuracy and symmetry of the
> timestamps that the server has. In absolute terms it's usually few
> microseconds or tens of microseconds. In relative terms, it can be up
> to about two orders of magnitude.
> 
> Basically, it makes the accuracy of NTP comparable to PTP. In networks
> where switches/routers support PTP, it's worse as the devices
> typically don't have a HW support for NTP. In networks without PTP
> support it's better than PTP, because PTP is very sensitive to
> variability in network delay. Of course, it also depends on the
> implementation.
> 
> If you just want to see an example of a difference in ideal
> conditions, here are two sourcestats comparing the same server to a
> PTP clock. In the first one it's using the basic mode, in the second
> it's using the interleaved mode.
> 
> Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
> ==============================================================================
> PTP                         8   5     7     -0.000      0.003     -0ns     3ns
> 192.168.33.1               21  13    22     +0.004      0.031  +3175ns   240ns
> 
> Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
> ==============================================================================
> PTP                        13   7    12     -0.000      0.001     -0ns     4ns
> 192.168.33.1               22  10    21     +0.001      0.001    +18ns     9ns
> 
> So, in this case the stability of NTP measurements improved by a
> factor of about 20 and the accuracy relative to the PTP clock by a
> factor of about 150.
> 
>> If you're testing this with chrony I would imagine the results would be
>> slightly different from ntpd, as I don't believe chrony implements the
>> same internal algorithms and I don't know how significant that would be
>> to the results.
> 
> That shouldn't matter much if you just want to see a difference. The
> problem is that ntpd doesn't support hardware timestamping or kernel
> transmit timestamping. The interleaved mode makes a difference only if
> the server has a more accurate transmit timestamp after the
> transmission.
> 
>>> The receive and transmit timestamps in server's responses, which are
>>> copied to the origin timestamp in client requests, are fully under the
>>> control of the server. A server with a sane clock will not respond
>>> with a duplicate receive timestamp, so each client using interleaved
>>> mode sends a unique origin timestamp.
> 
>> I'm not talking about server responses, I'm talking about the server
>> properly matching stored timestamps when there are an increasing number
>> of clients behind a single IP.
> 
> Well, unique receive timestamps in server responses is what enables
> the server to properly match them with stored transmit timestamps. It
> doesn't matter how many clients are there behind a single IP. Each
> should get a unique timestamp from the server.
> 
>> This brings up the question of what happens if a server response is lost
>> - in that case the server will have sent its interleaved reply and will
>> switch the client's saved transmit/origin timestamp.  When the client
>> doesn't get the response, it will then send the next packet with the
>> previous origin timestamp, the one the server no longer has because it
>> already replied using that one?
> 
> It depends on the implementation. It may drop the timestamp to which
> it already responded to, but it doesn't have to. The python example
> would keep it, chronyd would not.
> 
>> Yes, and one way to manage this is with a capped queue length, and
>> another way to manage it is with client limits (if there is a reasonable
>> way to manage this).  My point goes more towards the belief that there
>> is benefit to removing old/expired timestamps from the queue sooner
>> rather than later.  So it may make sense to keep a timestamp for, say, 2
>> seconds past 2x(poll interval), as we don't know if the client might
>> decide to increase its poll interval based on whatever it learned from
>> our most recent response.
> 
> It might work, but it will need more memory per client. And that
> memory could be used for more timestamps and keep it simpler. One pair
> of timestamps is 16 bytes, or just 12 bytes in an optimized
> implementation. So, even a little bit of extra state can decrease the
> number of concurrent clients significantly.
> 
> In any case, I think we would need to specify a requirement for
> interleaved clients to not zero the "poll" field.
> 

-- 
Harlan Stenn <[e-mail]>
http://networktimefoundation.org - be a member!