Re: [babel] Iotdir telechat review of draft-ietf-babel-rtt-extension-05

[Pascal Thubert <[e-mail]>]

Hello Juliusz

Please see below

cheers,

Pascal

Le 17/02/2024 à 20:22, Juliusz Chroboczek a écrit :
>> an applicability statement of the various possibilities would be useful
>> in the future.  Could be a paper or an RFC.  AT least it would make
>> sense to have an applicability section here.  For instance, IOT may
>> experience large and asymetric delays
> Section 1 describes the conditions in which we know the protocol to be
> applicable: cross-continent overlay networks.  At this time, we are not
> proposing this protocol for IoT-style applications, which have completely
> different time-scales than cross-continent overlay networks.
>
> We encourage people interested in IoT to borrow from our ideas, we'll take
> it as a compliment.

Right; I'm not sure the conditions are that clear though. I was looking 
for words that the latency has to be stable and measurable: limited 
jitter/latency ratio - which is not necessarily the case of IoT- and 
values order(s) of magnitude above than the timing precision.

.
>
>> Note that for the given example speed of light will certainly have
>> measurable effects. But going to Orleans and back may be hidden inside
>> e.g., wireless delays.
> Yes, Orléans is at 500µs from Paris, way below the recommended value of
> rtt-min (Section 4.2), and will therefore be classified as a local link.

Meaning that the algo is not meant to make that difference. This is  
again an interesting thing for the applicability (applies where the $ 
cost is not critical)

>
>> I'm effectively concerned with the effect of buffer bloats which could
>> create oscillations exactly like early ARPNET load-based metric.
> So are we, and that's what the whole of Section 4 is about.

I had the knee jerk reaction because the text there did not say that. 
Maybe a sentence there would solve my reader panic

> The bit that you quoted explicitly references Section 3.4.2 of RFC 8966.
> Are you suggesting that we need to repeat the contents of RFC 8966 here?
> Please clarify.

just one sentence saying what IHU is saves the novice reader going to 
the main RFC. In this case though, I expect the expected reader knows 
perfectly what IHUs are, so maybe just expanding IHU does the trick.


>> Ref IEEE 1588? there are many profiles for it; maybe this work could
>> show as one.
> I'm not sure what you're suggesting exactly.  Please clarify.

You are defining an operation that seems to inherit from 1588. This 
inheritance is typically expressed as a profile. There are things that 
come with the profile and are to be explicit to make the story complete. 
But maybe it's overkill. I leave to your discretion to look at it and 
mention it or not.

Important to indicate which time stamps are used (eg where in the stack 
is t1
>> measured). Do we measure the latency inside the sender meaning that the time
>> stamp is that of the software above, or do we measure stating at MAX enqueue,
>> or starting at PHY XMIT?
> The implementation note in Section 3.4 recommends timestamping just before
> the call to sendmsg.  I'll see if I can add some normative language to
> this effect.

please add words with reference to 3.4 and  explain that the latency 
below the API is dependent on the link load but not the system load


>> For short distance / high precision as claimed in the introduction,
> There is no such claim in the introduction.  The paragraph that confused
> you was only meant to point at potentially interesting further reasearch,
> I'll remove it.

"In this document, we specify an extension to the Babel routing protocol 
that enables*precise* measurement of the round-trip time (RTT) of a 
link, and allows its usage in metric computation. Since this causes a 
negative feedback loop, special care is needed to ensure that the 
resulting network is reasonably stable (Section 4 
<https://www.ietf.org/archive/id/draft-ietf-babel-rtt-extension-05.html#route-selection>).

"

maybe the term precise there is the trouble; also maybe the "stable" 
word should qualify the link loads and their respective latency as 
opposed to vaguely the network

>>     In principle, this algorithm is inaccurate in the presence of clock
>>     drift (i.e., when A's and B's clocks are running at different
>>     frequencies).  However, t2' - t1' is usually on the order of seconds,
>>     and significant clock drift is unlikely to happen at that time scale.
>> back to applicability of the work. I believe some expectations on the clock
>> drift vs RTT can be made for modern hardware. Nodes have an idea of which clock
>> they use and what drift they have. The draft could recommend that the clocking
>> error be 2 orders of magnitude less than the RTTs that the protocol measures,
>> else the measurement cannot be trusted.
> With the default parameters used by Babel, the time between Hello and IHU
> is 2s on average.  A cheap crystal oscillator, such as used in consumer
> electronics, has a typical drift of 10ppm (30ppm worst case), leading to an
> error of 20µs (60µs worst case).
>
> I also don't see where the "two orders of magnitude" figure comes from.
> The goal of this protocol is to disambiguate between local and distant
> routes, not to accurately determine the physical properties of links.

"two orders of magnitude" means that the error is in the percent range or below, meaning that they can be ignored for your purpose.
For the crystals I agree there's little issue, little point having the text above.
The troubling error does not come from the drift between message and response, but from the local system loads that are added to the real transfer latency, like how long an incoming message will be queued before processing, or, if 2 routes are over the same Wi-Fi adapter, how the WI-Fi MAC-PHY queues will impact differential the 2 measurements. Are both errors also below in the % range or below? Probably with RTT min.

>> Back to my earlier question of which step in the stack is relevant for
>> this measurement. Surelly any step that is dependent on the load of this
>> system (variable but independent of the link being used) as opposed to
>> the load to the transmission should be omitted.
> So if a router is loaded, we'll get an extra 1ms jitter.  This is not
> likely to impact route selection, and even if it does, it will merely
> cause the protocol to route around overloaded routers.

These are the words I was looking for. That jitter will be ignored 
because (hysteresis, RTT min, smoothing,  etc...)


>>     Second, using the RTT signal for route selection gives rise to a
>>     negative feedback loop: when a route has a low RTT, it is deemed to
>>     be more desirable, which causes it to be used for more data traffic,
>>     which may lead to congestion, which in turn increases the RTT.
>>     Without some form of hysteresis, using RTT for route selection would
>>     lead to oscillations between parallel routes, which might lead to
>>     packet reordering and negatively affect upper-layer protocols (such
>>     as TCP).
>>
>> I believe this discussion should be seen earlier in the text, eg in the
>> introduction (not the solution but at least that the issue exists and is
>> addressed in the protocol). See my early comment on ARPANET.
> I most respectfully disagree.  This document is structured in two parts,
> a first part that defines a subprotocol that produces a continuous stream
> of RTT samples, and a second part that describes an algorithm to extract
> from that stream information that is useful for route selection.

OK with the subprotocol thing.  My ask is really that the reader knows 
early that you handle the oscillations and reasonable jitter (latency 
variations).



>> 4.3.  Hysteresis
>>
>>     Even after applying a bounded mapping from smoothed RTT to a cost
>>     value, the cost may fluctuate when a link's RTT is between rtt-min
>>     and rtt-max.  This is effectively mitigated by using a robust
>>     hysteresis algorithm, such as the one described in Appendix A.3 of
>>     [RFC8966].
>>
>> if this is what solves the oscillation issue please mention it,
> No, it's more complex than that.  There are three disctinct mechanisms
> that collaborate to avoid oscilliations.  The smoothing in Section 4.1
> avoids oscillations due to outliers.  The non-linear mapping from RTT to
> cost described in Section 4.2 avoids oscillations for good links (below
> rtt-min) and for bad links (above rtt-max).  Hysteresis is a last-resort
> mechanism that mitigates the issue for links between rtt-min and rtt-max.
>
> I've just re-read Section 4, and I think it's clear enough.  Please let me
> know if you have suggestions to make it better.


Do not change section 4 for me, but maybe the text above is exactly the 
fwd ref I'm looking for early in the text


>> Maybe discuss the consequences of a MIM that modifies the values eg to
>> discourage Paris to Paris and cause routing via Tokyo?
> If you're not using cryptographic signatures, then a MITM has easier ways
> to redirect traffic.  See Section 6 of RFC 8966.

Great, please add that to the security section.

Many thanks!

Pascal


>
> -- Juliusz