Re: [Detnet] Mirja Kühlewind's Discuss on draft-ietf-detnet-architecture-11: (with DISCUSS and COMMENT)

[János Farkas <[e-mail]>]

Thank you Mirja!
We are implementing the changes in the next revision of the draft.
Janos

On 3/5/2019 11:28 AM, Mirja Kuehlewind (IETF) wrote:
> Hi Janos,
>
> sorry for my late'ish reply. Your changes look good to me. Thanks!
>
> Mirja
>
>
>
>> Am 26.02.2019 um 13:45 schrieb János Farkas <[e-mail]>:
>>
>> Hi Mirja,
>>
>> Thank you for your review!
>>
>> Please see in-line changes suggested to address your comments.
>>
>>
>> On 2/20/2019 5:24 PM, Lou Berger wrote:
>>> Hi Mirja,
>>>
>>>      See below.
>>>
>>> On 2/20/2019 11:06 AM, Mirja Kuehlewind (IETF) wrote:
>>>> Hi Lou,
>>>>
>>>> I’m happy to see that my concerns seem to be cover. So maybe this is mostly a point about wording. Please see below.
>>>>
>>>>> Am 20.02.2019 um 15:46 schrieb Lou Berger <[e-mail]>:
>>>>>
>>>>> Hi Mirja,
>>>>>
>>>>> I'm responding as doc shepherd (and chair). Thank you for your comments.  Please see below for specific responses.
>>>>>
>>>>> On 2/20/2019 8:05 AM, Mirja Kühlewind wrote:
>>>>>> Mirja Kühlewind has entered the following ballot position for
>>>>>> draft-ietf-detnet-architecture-11: Discuss
>>>>>>
>>>>>> When responding, please keep the subject line intact and reply to all
>>>>>> email addresses included in the To and CC lines. (Feel free to cut this
>>>>>> introductory paragraph, however.)
>>>>>>
>>>>>>
>>>>>> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
>>>>>> for more information about IESG DISCUSS and COMMENT positions.
>>>>>>
>>>>>>
>>>>>> The document, along with other ballot positions, can be found here:
>>>>>> https://datatracker.ietf.org/doc/draft-ietf-detnet-architecture/
>>>>>>
>>>>>>
>>>>>>
>>>>>> ----------------------------------------------------------------------
>>>>>> DISCUSS:
>>>>>> ----------------------------------------------------------------------
>>>>>>
>>>>>> Thanks for addressing the tsv-art review comments  (and big thanks to Michael
>>>>>> and David!) and all the work done so far! I think the document is in good shape
>>>>>> and I only have one minor comment that I would like to see addressed/more
>>>>>> explicitly spelled out. However, this should be done quickly with potentially
>>>>>> 2-3 small changes in the draft. See below.
>>>>> The input from Michael and David certainly led to a much improved document.  If you can help the authors with some specific text suggestions that would be most helpful.
>>>>>
>>>>>
>>>>>> Given that DetNet traffic is often assumed to be not congestion controlled, it
>>>>>> is important that there is also some network function that makes sure the
>>>>>> source traffic stays within the requested bandwidth limit in-order to protect
>>>>>> non-Detnet traffic. This is to some extended discussed in section 3.3.2 but I
>>>>>> think it should be more clearly spelled out that this would require a rate
>>>>>> limiting function at each DetNet source/relay (tunnel ingress). Currently sec
>>>>>> 3.3.2 says:
>>>>>>
>>>>>> "Filters and policers should be used in a DetNet network to
>>>>>>      detect if DetNet packets are received on the wrong interface, or at
>>>>>>      the wrong time, or in too great a volume."
>>>>>>
>>>>>> However, maybe this case of limiting non-congestion controlled traffic (in case
>>>>>> the source in not keeping to the limits on purpose/in order to cheat, because
>>>>>> it couldn't estimate the needed bandwidth requirement an better, or due to
>>>>>> timely fluctuations) could be explained more clearing and the respective
>>>>>> requirement to implement rate limiting could be state separately and more
>>>>>> strongly...?
>>>>> Section 3.2.1.1 days:
>>>>>
>>>>>     Ensuring adequate buffering requires, in turn, that the source, and
>>>>>     every DetNet node along the path to the destination (or nearly every
>>>>>     node, see Section 4.3.3) be careful to regulate its output to not
>>>>>     exceed the data rate for any DetNet flow, except for brief periods
>>>>>     when making up for interfering traffic.  Any packet sent ahead of its
>>>>>     time potentially adds to the number of buffers required by the next
>>>>>     hop DetNet node and may thus exceed the resources allocated for a
>>>>>     particular DetNet flow.
>>>>>
>>>>> Section 3.3.1 also says:
>>>>>
>>>>>     Starvation of non-DetNet traffic must be avoided, e.g., by traffic
>>>>>     policing functions (e.g., [RFC2475]).  Thus, the net effect of the
>>>>>     presence of DetNet flows in a network on the non-DetNet flows is
>>>>>     primarily a reduction in the available bandwidth.
>>>>>
>>>>> Is this not sufficient?  If not, can you suggest some additional language?
>>>> My question is mostly if you only need to protect _complete_ starvation or any over-use by DetNet flows and how you would do that. If the answer is to rate limit the DetNet flows at ingress to the allocated bandwidth, that would completely address my concern. However note that buffer dimensioning and rate limiting is not the same. Rate limiting is usually a scheduling question, e.g. using a token bucket scheme or a fixed non-work conserving scheduler. Therefore it would be nice to add one additional sentence in section 3.2 that spells that out directly. e.g. something like:
>>>>
>>>> „In order to protect non-DetNet traffic from potentially misbehaving DetNet-Traffic sources, rate limiting, e.g. using a token bucket scheme, at each tunnel ingress must be applied.“
>>>> I’m sure you can adapt the wording to make it more DetNet-like language and maybe you also want to add another sentence explaining the problem, please feel free to do so but that’s more or less what I’m looking for.
>>> I do agree that we seem to be having a terminology issue here - and that we are on the same page with what should happen.  The terminology I've used/seen in past RFCs has usually used the terms 'shaping' and 'policing' - and these are what we've been using within the WG.  For example, both of these terms are used in RFC2475 but 'rate limiting' is not. Would it work for you for the draft to use/adapt the rest of your language, but keep 'by traffic policing functions (e.g., [RFC2475])' rather than 'rate limiting, e.g. using a token bucket scheme,' ?
>> We could extend the 2nd paragraph in 3.2.1.1 (https://tools.ietf.org/html/draft-ietf-detnet-architecture-11#section-3.2.1.1) to capture this comment:
>>
>> OLD:
>>              Ensuring adequate buffering requires, in turn, that the source, and every DetNet
>>              node along the path to the destination (or nearly every node, see
>>              <xref target="Incomplete"/>) be careful to regulate its output to not exceed the
>>              data rate for any DetNet flow, except for brief periods when making up for
>>              interfering traffic.  Any packet sent ahead of its time potentially adds to the
>>              number of buffers required by the next hop DetNet node and may thus exceed the resources
>>              allocated for a particular DetNet flow.
>>
>> NEW:
>>
>>              Ensuring adequate buffering requires, in turn, that the source, and every DetNet
>>              node along the path to the destination (or nearly every node, see
>>              <xref target="Incomplete"/>) be careful to regulate its output to not exceed the
>>              data rate for any DetNet flow, except for brief periods when making up for
>>              interfering traffic.  Any packet sent ahead of its time potentially adds to the
>>              number of buffers required by the next hop DetNet node and may thus exceed the resources
>>              allocated for a particular DetNet flow. Furthermore, rate limiting, e.g., using traffic
>>              policing and shaping functions, e.g., <xref target="RFC2475"/>, at the ingress of the
>>              DetNet domain must be applied. This is needed for meeting the requirements of DetNet
>>              flows as well as for protecting non-DetNet traffic from potentially misbehaving DetNet
>>              traffic sources.
>>
>>
>> Furthermore, replace the two occurrences of "policing" with "policing and shaping":
>>
>> OLD:
>>     Starvation of non-DetNet traffic must be avoided, e.g., by traffic
>>     policing functions (e.g., [RFC2475]).
>>
>> NEW:
>>     Starvation of non-DetNet traffic must be avoided, e.g., by traffic
>>     policing and shaping functions (e.g., [RFC2475]).
>>
>>
>> OLD:
>>     Examples of such techniques include traffic policing
>>     functions (e.g., [RFC2475]) and separating flows into per-flow rate-
>>     limited queues.
>>
>> NEW:
>>     Examples of such techniques include traffic policing and shaping
>>     functions (e.g., [RFC2475]) and separating flows into per-flow rate-
>>     limited queues.
>>
>>>
>>>>>> One related comment on this sentence in Sec 3.1:
>>>>>>
>>>>>> "As DetNet provides allocated resources (including provisioned capacity)
>>>>>>      to DetNet flows the use of transport layer congestion control
>>>>>>      [RFC2914] by App-flows is explicitly not required."
>>>>>>
>>>>>> I guess congestion control should still be a requirement if the App-flow also
>>>>>> passes not-DetNet-aware segments of the path, e.g. maybe the first hop.
>>>>> This point is addressed in section 3.3.2
>>>>>
>>>>>     Note that the sending of App-flows that do not use transport layer
>>>>>     congestion control per [RFC2914] into a network that is not
>>>>>     provisioned to handle such DetNet traffic has to be treated as a
>>>>>     fault and prevented.
>>>>>
>>>>>> Usually
>>>>>> use of congestion control for application limited flows is also not a problem
>>>>>> if sufficient bandwidth is available. Also note that, as I stated above, the
>>>>>> important part for not requiring congestion control is actually not only that
>>>>>> resources are allocated but also that rate limiting is in place to make sure
>>>>>> resources usage cannot be exceeded above the reserved allocation. Maybe this
>>>>>> sentence could also be further clarified in the draft...?
>>>>> Is the above sufficient?  If not, can you suggest some additional language?
>>>> Maybe we can just extend the above statement a bit in order to be crystal clear about this, e.g.:
>>>>
>>>> "As DetNet flow are assumed to be rate-limited and DetNet is designed to provide sufficient allocated resources (including provisioned capacity), the use of transport layer congestion control [RFC2914] for end-to-end DetNet flows is not required, however, if resources are allocated appropriately, use of congestion control should not impact transmission negatively.“
>>> Great!  (I'll leave it to the authors to do the final wording adaptation)
>> "App-flow" seems to be better than "end-to-end DetNet flow". The change with that would be:
>>
>> OLD:
>>     As
>>     DetNet provides allocated resources (including provisioned capacity)
>>     to DetNet flows the use of transport layer congestion control
>>     [RFC2914] by App-flows is explicitly not required.
>>
>> NEW:
>>              As
>>              DetNet flows are assumed to be rate-limited and DetNet is designed
>>              to provide sufficient allocated resources (including provisioned
>>              capacity), the use of transport layer congestion control [RFC2914]
>>              for App-flows is not required; however, if resources are allocated
>>              appropriately, use of congestion control should not impact
>>              transmission negatively.
>>
>>>>>> And then one more small comment that is also related. Sec 3.2.2.2 says:
>>>>>>
>>>>>> "If packet replication and elimination is used over paths with
>>>>>>      resource allocation (Section 3.2.1), ..."
>>>>>>
>>>>>> My assumption was that all DetNet traffic is send over pre-allocated
>>>>>> resources...? If that is not true that has implication on congestion control
>>>>>> and needs some additional considerations. Can you please confirm that and maybe
>>>>>> clarify in the draft! Thanks!
>>>>> The intent was to cover sending over non-pre-allocated paths as a fault case, and cover it in section 3.3.2.
>>>> If that is the case, maybe we can just removed the half-sentence I cited above?
>>> WFM!
>> The sentence after removing the text as you suggest:
>>
>> OLD:
>>     If packet replication and elimination is used over paths with
>>     resource allocation (Section 3.2.1), and member flows that take
>>     different-length paths through the network are combined, a merge
>>     point may require extra buffering to equalize the delays over the
>>     different paths.
>>
>> NEW:
>>     If member flows that take different-length paths through the
>>     network are combined, a merge point may require extra buffering
>>     to equalize the delays over the different paths.
>>
>>
>> Thank you!
>> Janos
>>
>>
>>> Thank you!
>>>
>>>> Thanks!
>>>> Mirja
>>>>
>>>>> In addition to the previously quoted text, the section also says:
>>>>>
>>>>>     PRF generated DetNet member flows also need to
>>>>>     be treated as not using transport layer congestion control even if
>>>>>     the original App-flow supports transport layer congestion control
>>>>>     because PREOF can remove congestion indications at the PEF and
>>>>>     thereby hide such indications (e.g., drops, ECN markings, increased
>>>>>     latency) from end systems.
>>>>>
>>>>> Does this address your concern?
>>>>>> ----------------------------------------------------------------------
>>>>>> COMMENT:
>>>>>> ----------------------------------------------------------------------
>>>>>>
>>>>>> I agree with Alexey and Benjamin that this document should be informational.
>>>>>> Informational documents can also have IETF consensus, so that cannot be the
>>>>>> reason to go for PS. However, this document does not specify a protocol or any
>>>>>> requirements that are mandatory to implement for interoperability and therefore
>>>>>> should not be PS.
>>>>>>
>>>>> I defer to the IESG and AD on this...
>>>>>
>>>>> Thank you!
>>>>>
>>>>> Lou
>>>>>
>>>>>> _______________________________________________
>>>>>> detnet mailing list
>>>>>> [e-mail]
>>>>>> https://www.ietf.org/mailman/listinfo/detnet