Re: [Ntp] NTP over PTP

[Miroslav Lichvar <[e-mail]>]

On Wed, Jun 30, 2021 at 03:42:53PM +0200, Heiko Gerstung wrote:
> > Well, yes, but the delay measurement is separate from the offset
> > measurement. If you can log and plot the offsets measured by PTP and
> > NTP in a network without PTP support, you will see how the
> > distribution is different due to the different calculation.
> 
> This depends, as pointed out earlier, on the network itself. If you have highly dynamic network paths where traffic patterns change quickly and therefore result in wildly changing queueing and forwarding delays, you will see slightly different delay distribution in NTP and PTP logs. In less dynamic environments, you will probably not notice something like that at all.

I'm referring to the distribution of the offset as measured by PTP
and NTP. Maybe this plot will make it more clear.

https://fedorapeople.org/~mlichvar/tmp/ntp-ptp-offset.png

It shows offset over time as calculated by NTP and PTP in identical
conditions. Hopefully you can see that the offset on the top has a
symmetric distribution and one on the bottom does not. With the PTP
offset calculation (same as in the NTP broadcast mode), the
distribution of the network delay in the server->client direction
transfers to the distribution of the offset. In the NTP client/server
mode the offset is calculated from the sum of delays in both
directions. It's not difficult to guess what is easier to work with
when controlling a clock.

> And in networks with partial or full on-path support, you will see that PTP is performing great with its inferior delay measurement approach. 

With full on-path support, yes, it performs great. That's what it was
designed for. Without full support on a mostly idle network or in a
network where PTP messages are prioritized, it can still work ok. But
as soon as PTP messages start waiting in longer queues, it falls apart
very quickly. NTP can generally perform well in worse conditions as it
measures the offset and delay at the same time, while PTP assumes the
delay didn't change when it measures the offset.

Of course, there are differences between implementations. A more
advanced PTP implementation can certainly perform better than a less
advanced NTP implementation. PTP implementations can also ignore the
PTP specification and use the NTP approach.

> >> Yes, there are implementations which take that into account by applying
> >> static correction values to compensate for link speed asymmetry. I believe this
> >> also affects NTP, but in most cases hundreds of nanoseconds are not a problem
> >> for applications relying on NTP synchronization.
> > 
> > NTP is much less impacted as it timestamps the end of the reception.
> > A software timestamp is captured after the packet is received.
> 
> That is a very confusing argument. 

I'm talking here specifically about the error that is caused by server
and client being on different link speeds, not other sources of error
due to processing delays, etc. NTP using trailing timestamps (which
requires transposition of hardware timestamps) is another reason why
NTP should be preferred over PTP in networks that don't have a full
on-path support.

> When we connect one of our grandmaster clocks (PTP server) to one of our PTP client devices over a direct crossover cable or fiber link, we can measure an offset of less than 20ns between the PPS generated by the GNSS synchronized master clock and the client. 

I see the same with NTP here.

> > Isn't that an issue for both NTP and PTP unicast using high rate sync
> > and delay requests?
> 
> Yes, but unicast PTP servers with hardware time stamping are typically commercial products with dedicated network hardware to support the large number of timestamps that have to be stored and processed.

Would this hardware not work with NTS-over-PTP, but work with
NTS4UPTP?

> The i210 has a time stamping queue length of 4, i.e. it stores a maximum of 4 timestamps in a ring buffer. A unicast PTP server with 10 clients running at 128 sync and delay pkt/s will generate 1,280 time stamps per second. Even on some serious hardware there will be a significant number of timestamps that are lost due to queue overruns.  

I think you are confusing the I210 with something else. It can
timestamp all received packets, at any rate.

Its performance in a server is limited by TX timestamping. It can
timestamp only one packet at a time. But the maximum rate is
definitely higher than 1280 per second. In my tests it's about 35000
TX timestamps per second. That's with the stock Linux driver which
seems to rely on an interrupt. I think it could be modified to poll
the timestamp register for a much higher rate of timestamps. In any
case, I'd say that is pretty good for a $50 NIC.

> > I think the best approach is for the hardware to timestamp all packets
> > as many NICs already do. The problem is with existing hardware that
> > cannot do that. I agree it doesn't look great when you have to run NTP
> > over PTP, but that seems to be the only way to get the timestamping
> > working on this hardware.
> 
> I believe there should be a standard that adds a hardware timestamp to the end of every Ethernet frame. It requires a NIC vendor to implement a hardware clock and a time stamping engine into their silicon. The bandwidth between the NIC and the OS layer (driver) has to accommodate the extra data (but you could reduce the MTU of course). 

That's how it works with the I210 and other NICs, except the timestamp
is before the frame, not after.

-- 
Miroslav Lichvar