Re: [Detnet] Mirja Kühlewind's Discuss on draft-ietf-detnet-architecture-11: (with DISCUSS and COMMENT)

[János Farkas <[e-mail]>]

Hi Mirja,

We believe that we have addressed your comments in the most recent 
revision: https://tools.ietf.org/html/draft-ietf-detnet-architecture-12. 
(https://mailarchive.ietf.org/arch/msg/detnet/utVL9ZVGcOeGtRIASRFx5WT_ErM)

Please let us know what else you would like to see done before you clear 
your DISCUSS.

I/we would be happy to meet with you this week if there is anything you 
would like to discuss.

Regards,
Janos


On 3/5/2019 11:30 AM, János Farkas wrote:
> Thank you Mirja!
> We are implementing the changes in the next revision of the draft.
> Janos
>
> On 3/5/2019 11:28 AM, Mirja Kuehlewind (IETF) wrote:
>> Hi Janos,
>>
>> sorry for my late'ish reply. Your changes look good to me. Thanks!
>>
>> Mirja
>>
>>
>>
>>> Am 26.02.2019 um 13:45 schrieb János Farkas 
>>> <[e-mail]>:
>>>
>>> Hi Mirja,
>>>
>>> Thank you for your review!
>>>
>>> Please see in-line changes suggested to address your comments.
>>>
>>>
>>> On 2/20/2019 5:24 PM, Lou Berger wrote:
>>>> Hi Mirja,
>>>>
>>>>      See below.
>>>>
>>>> On 2/20/2019 11:06 AM, Mirja Kuehlewind (IETF) wrote:
>>>>> Hi Lou,
>>>>>
>>>>> I’m happy to see that my concerns seem to be cover. So maybe this 
>>>>> is mostly a point about wording. Please see below.
>>>>>
>>>>>> Am 20.02.2019 um 15:46 schrieb Lou Berger <[e-mail]>:
>>>>>>
>>>>>> Hi Mirja,
>>>>>>
>>>>>> I'm responding as doc shepherd (and chair). Thank you for your 
>>>>>> comments.  Please see below for specific responses.
>>>>>>
>>>>>> On 2/20/2019 8:05 AM, Mirja Kühlewind wrote:
>>>>>>> Mirja Kühlewind has entered the following ballot position for
>>>>>>> draft-ietf-detnet-architecture-11: Discuss
>>>>>>>
>>>>>>> When responding, please keep the subject line intact and reply 
>>>>>>> to all
>>>>>>> email addresses included in the To and CC lines. (Feel free to 
>>>>>>> cut this
>>>>>>> introductory paragraph, however.)
>>>>>>>
>>>>>>>
>>>>>>> Please refer to 
>>>>>>> https://www.ietf.org/iesg/statement/discuss-criteria.html
>>>>>>> for more information about IESG DISCUSS and COMMENT positions.
>>>>>>>
>>>>>>>
>>>>>>> The document, along with other ballot positions, can be found here:
>>>>>>> https://datatracker.ietf.org/doc/draft-ietf-detnet-architecture/
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------- 
>>>>>>>
>>>>>>> DISCUSS:
>>>>>>> ---------------------------------------------------------------------- 
>>>>>>>
>>>>>>>
>>>>>>> Thanks for addressing the tsv-art review comments (and big 
>>>>>>> thanks to Michael
>>>>>>> and David!) and all the work done so far! I think the document 
>>>>>>> is in good shape
>>>>>>> and I only have one minor comment that I would like to see 
>>>>>>> addressed/more
>>>>>>> explicitly spelled out. However, this should be done quickly 
>>>>>>> with potentially
>>>>>>> 2-3 small changes in the draft. See below.
>>>>>> The input from Michael and David certainly led to a much improved 
>>>>>> document.  If you can help the authors with some specific text 
>>>>>> suggestions that would be most helpful.
>>>>>>
>>>>>>
>>>>>>> Given that DetNet traffic is often assumed to be not congestion 
>>>>>>> controlled, it
>>>>>>> is important that there is also some network function that makes 
>>>>>>> sure the
>>>>>>> source traffic stays within the requested bandwidth limit 
>>>>>>> in-order to protect
>>>>>>> non-Detnet traffic. This is to some extended discussed in 
>>>>>>> section 3.3.2 but I
>>>>>>> think it should be more clearly spelled out that this would 
>>>>>>> require a rate
>>>>>>> limiting function at each DetNet source/relay (tunnel ingress). 
>>>>>>> Currently sec
>>>>>>> 3.3.2 says:
>>>>>>>
>>>>>>> "Filters and policers should be used in a DetNet network to
>>>>>>>      detect if DetNet packets are received on the wrong 
>>>>>>> interface, or at
>>>>>>>      the wrong time, or in too great a volume."
>>>>>>>
>>>>>>> However, maybe this case of limiting non-congestion controlled 
>>>>>>> traffic (in case
>>>>>>> the source in not keeping to the limits on purpose/in order to 
>>>>>>> cheat, because
>>>>>>> it couldn't estimate the needed bandwidth requirement an better, 
>>>>>>> or due to
>>>>>>> timely fluctuations) could be explained more clearing and the 
>>>>>>> respective
>>>>>>> requirement to implement rate limiting could be state separately 
>>>>>>> and more
>>>>>>> strongly...?
>>>>>> Section 3.2.1.1 days:
>>>>>>
>>>>>>     Ensuring adequate buffering requires, in turn, that the 
>>>>>> source, and
>>>>>>     every DetNet node along the path to the destination (or 
>>>>>> nearly every
>>>>>>     node, see Section 4.3.3) be careful to regulate its output to 
>>>>>> not
>>>>>>     exceed the data rate for any DetNet flow, except for brief 
>>>>>> periods
>>>>>>     when making up for interfering traffic.  Any packet sent 
>>>>>> ahead of its
>>>>>>     time potentially adds to the number of buffers required by 
>>>>>> the next
>>>>>>     hop DetNet node and may thus exceed the resources allocated 
>>>>>> for a
>>>>>>     particular DetNet flow.
>>>>>>
>>>>>> Section 3.3.1 also says:
>>>>>>
>>>>>>     Starvation of non-DetNet traffic must be avoided, e.g., by 
>>>>>> traffic
>>>>>>     policing functions (e.g., [RFC2475]).  Thus, the net effect 
>>>>>> of the
>>>>>>     presence of DetNet flows in a network on the non-DetNet flows is
>>>>>>     primarily a reduction in the available bandwidth.
>>>>>>
>>>>>> Is this not sufficient?  If not, can you suggest some additional 
>>>>>> language?
>>>>> My question is mostly if you only need to protect _complete_ 
>>>>> starvation or any over-use by DetNet flows and how you would do 
>>>>> that. If the answer is to rate limit the DetNet flows at ingress 
>>>>> to the allocated bandwidth, that would completely address my 
>>>>> concern. However note that buffer dimensioning and rate limiting 
>>>>> is not the same. Rate limiting is usually a scheduling question, 
>>>>> e.g. using a token bucket scheme or a fixed non-work conserving 
>>>>> scheduler. Therefore it would be nice to add one additional 
>>>>> sentence in section 3.2 that spells that out directly. e.g. 
>>>>> something like:
>>>>>
>>>>> „In order to protect non-DetNet traffic from potentially 
>>>>> misbehaving DetNet-Traffic sources, rate limiting, e.g. using a 
>>>>> token bucket scheme, at each tunnel ingress must be applied.“
>>>>> I’m sure you can adapt the wording to make it more DetNet-like 
>>>>> language and maybe you also want to add another sentence 
>>>>> explaining the problem, please feel free to do so but that’s more 
>>>>> or less what I’m looking for.
>>>> I do agree that we seem to be having a terminology issue here - and 
>>>> that we are on the same page with what should happen.  The 
>>>> terminology I've used/seen in past RFCs has usually used the terms 
>>>> 'shaping' and 'policing' - and these are what we've been using 
>>>> within the WG.  For example, both of these terms are used in 
>>>> RFC2475 but 'rate limiting' is not. Would it work for you for the 
>>>> draft to use/adapt the rest of your language, but keep 'by traffic 
>>>> policing functions (e.g., [RFC2475])' rather than 'rate limiting, 
>>>> e.g. using a token bucket scheme,' ?
>>> We could extend the 2nd paragraph in 3.2.1.1 
>>> (https://tools.ietf.org/html/draft-ietf-detnet-architecture-11#section-3.2.1.1) 
>>> to capture this comment:
>>>
>>> OLD:
>>>              Ensuring adequate buffering requires, in turn, that the 
>>> source, and every DetNet
>>>              node along the path to the destination (or nearly every 
>>> node, see
>>>              <xref target="Incomplete"/>) be careful to regulate its 
>>> output to not exceed the
>>>              data rate for any DetNet flow, except for brief periods 
>>> when making up for
>>>              interfering traffic.  Any packet sent ahead of its time 
>>> potentially adds to the
>>>              number of buffers required by the next hop DetNet node 
>>> and may thus exceed the resources
>>>              allocated for a particular DetNet flow.
>>>
>>> NEW:
>>>
>>>              Ensuring adequate buffering requires, in turn, that the 
>>> source, and every DetNet
>>>              node along the path to the destination (or nearly every 
>>> node, see
>>>              <xref target="Incomplete"/>) be careful to regulate its 
>>> output to not exceed the
>>>              data rate for any DetNet flow, except for brief periods 
>>> when making up for
>>>              interfering traffic.  Any packet sent ahead of its time 
>>> potentially adds to the
>>>              number of buffers required by the next hop DetNet node 
>>> and may thus exceed the resources
>>>              allocated for a particular DetNet flow. Furthermore, 
>>> rate limiting, e.g., using traffic
>>>              policing and shaping functions, e.g., <xref 
>>> target="RFC2475"/>, at the ingress of the
>>>              DetNet domain must be applied. This is needed for 
>>> meeting the requirements of DetNet
>>>              flows as well as for protecting non-DetNet traffic from 
>>> potentially misbehaving DetNet
>>>              traffic sources.
>>>
>>>
>>> Furthermore, replace the two occurrences of "policing" with 
>>> "policing and shaping":
>>>
>>> OLD:
>>>     Starvation of non-DetNet traffic must be avoided, e.g., by traffic
>>>     policing functions (e.g., [RFC2475]).
>>>
>>> NEW:
>>>     Starvation of non-DetNet traffic must be avoided, e.g., by traffic
>>>     policing and shaping functions (e.g., [RFC2475]).
>>>
>>>
>>> OLD:
>>>     Examples of such techniques include traffic policing
>>>     functions (e.g., [RFC2475]) and separating flows into per-flow 
>>> rate-
>>>     limited queues.
>>>
>>> NEW:
>>>     Examples of such techniques include traffic policing and shaping
>>>     functions (e.g., [RFC2475]) and separating flows into per-flow 
>>> rate-
>>>     limited queues.
>>>
>>>>
>>>>>>> One related comment on this sentence in Sec 3.1:
>>>>>>>
>>>>>>> "As DetNet provides allocated resources (including provisioned 
>>>>>>> capacity)
>>>>>>>      to DetNet flows the use of transport layer congestion control
>>>>>>>      [RFC2914] by App-flows is explicitly not required."
>>>>>>>
>>>>>>> I guess congestion control should still be a requirement if the 
>>>>>>> App-flow also
>>>>>>> passes not-DetNet-aware segments of the path, e.g. maybe the 
>>>>>>> first hop.
>>>>>> This point is addressed in section 3.3.2
>>>>>>
>>>>>>     Note that the sending of App-flows that do not use transport 
>>>>>> layer
>>>>>>     congestion control per [RFC2914] into a network that is not
>>>>>>     provisioned to handle such DetNet traffic has to be treated as a
>>>>>>     fault and prevented.
>>>>>>
>>>>>>> Usually
>>>>>>> use of congestion control for application limited flows is also 
>>>>>>> not a problem
>>>>>>> if sufficient bandwidth is available. Also note that, as I 
>>>>>>> stated above, the
>>>>>>> important part for not requiring congestion control is actually 
>>>>>>> not only that
>>>>>>> resources are allocated but also that rate limiting is in place 
>>>>>>> to make sure
>>>>>>> resources usage cannot be exceeded above the reserved 
>>>>>>> allocation. Maybe this
>>>>>>> sentence could also be further clarified in the draft...?
>>>>>> Is the above sufficient?  If not, can you suggest some additional 
>>>>>> language?
>>>>> Maybe we can just extend the above statement a bit in order to be 
>>>>> crystal clear about this, e.g.:
>>>>>
>>>>> "As DetNet flow are assumed to be rate-limited and DetNet is 
>>>>> designed to provide sufficient allocated resources (including 
>>>>> provisioned capacity), the use of transport layer congestion 
>>>>> control [RFC2914] for end-to-end DetNet flows is not required, 
>>>>> however, if resources are allocated appropriately, use of 
>>>>> congestion control should not impact transmission negatively.“
>>>> Great!  (I'll leave it to the authors to do the final wording 
>>>> adaptation)
>>> "App-flow" seems to be better than "end-to-end DetNet flow". The 
>>> change with that would be:
>>>
>>> OLD:
>>>     As
>>>     DetNet provides allocated resources (including provisioned 
>>> capacity)
>>>     to DetNet flows the use of transport layer congestion control
>>>     [RFC2914] by App-flows is explicitly not required.
>>>
>>> NEW:
>>>              As
>>>              DetNet flows are assumed to be rate-limited and DetNet 
>>> is designed
>>>              to provide sufficient allocated resources (including 
>>> provisioned
>>>              capacity), the use of transport layer congestion 
>>> control [RFC2914]
>>>              for App-flows is not required; however, if resources 
>>> are allocated
>>>              appropriately, use of congestion control should not impact
>>>              transmission negatively.
>>>
>>>>>>> And then one more small comment that is also related. Sec 
>>>>>>> 3.2.2.2 says:
>>>>>>>
>>>>>>> "If packet replication and elimination is used over paths with
>>>>>>>      resource allocation (Section 3.2.1), ..."
>>>>>>>
>>>>>>> My assumption was that all DetNet traffic is send over 
>>>>>>> pre-allocated
>>>>>>> resources...? If that is not true that has implication on 
>>>>>>> congestion control
>>>>>>> and needs some additional considerations. Can you please confirm 
>>>>>>> that and maybe
>>>>>>> clarify in the draft! Thanks!
>>>>>> The intent was to cover sending over non-pre-allocated paths as a 
>>>>>> fault case, and cover it in section 3.3.2.
>>>>> If that is the case, maybe we can just removed the half-sentence I 
>>>>> cited above?
>>>> WFM!
>>> The sentence after removing the text as you suggest:
>>>
>>> OLD:
>>>     If packet replication and elimination is used over paths with
>>>     resource allocation (Section 3.2.1), and member flows that take
>>>     different-length paths through the network are combined, a merge
>>>     point may require extra buffering to equalize the delays over the
>>>     different paths.
>>>
>>> NEW:
>>>     If member flows that take different-length paths through the
>>>     network are combined, a merge point may require extra buffering
>>>     to equalize the delays over the different paths.
>>>
>>>
>>> Thank you!
>>> Janos
>>>
>>>
>>>> Thank you!
>>>>
>>>>> Thanks!
>>>>> Mirja
>>>>>
>>>>>> In addition to the previously quoted text, the section also says:
>>>>>>
>>>>>>     PRF generated DetNet member flows also need to
>>>>>>     be treated as not using transport layer congestion control 
>>>>>> even if
>>>>>>     the original App-flow supports transport layer congestion 
>>>>>> control
>>>>>>     because PREOF can remove congestion indications at the PEF and
>>>>>>     thereby hide such indications (e.g., drops, ECN markings, 
>>>>>> increased
>>>>>>     latency) from end systems.
>>>>>>
>>>>>> Does this address your concern?
>>>>>>> ---------------------------------------------------------------------- 
>>>>>>>
>>>>>>> COMMENT:
>>>>>>> ---------------------------------------------------------------------- 
>>>>>>>
>>>>>>>
>>>>>>> I agree with Alexey and Benjamin that this document should be 
>>>>>>> informational.
>>>>>>> Informational documents can also have IETF consensus, so that 
>>>>>>> cannot be the
>>>>>>> reason to go for PS. However, this document does not specify a 
>>>>>>> protocol or any
>>>>>>> requirements that are mandatory to implement for 
>>>>>>> interoperability and therefore
>>>>>>> should not be PS.
>>>>>>>
>>>>>> I defer to the IESG and AD on this...
>>>>>>
>>>>>> Thank you!
>>>>>>
>>>>>> Lou
>>>>>>
>>>>>>> _______________________________________________
>>>>>>> detnet mailing list
>>>>>>> [e-mail]
>>>>>>> https://www.ietf.org/mailman/listinfo/detnet
>
>