Re: [Detnet] Mirja Kühlewind's Discuss on draft-ietf-detnet-architecture-11: (with DISCUSS and COMMENT)

[János Farkas <[e-mail]>]

Hi Mirja,

Thank you for your review!

Please see in-line changes suggested to address your comments.


On 2/20/2019 5:24 PM, Lou Berger wrote:
> Hi Mirja,
>
>     See below.
>
> On 2/20/2019 11:06 AM, Mirja Kuehlewind (IETF) wrote:
>> Hi Lou,
>>
>> I’m happy to see that my concerns seem to be cover. So maybe this is 
>> mostly a point about wording. Please see below.
>>
>>> Am 20.02.2019 um 15:46 schrieb Lou Berger <[e-mail]>:
>>>
>>> Hi Mirja,
>>>
>>> I'm responding as doc shepherd (and chair). Thank you for your 
>>> comments.  Please see below for specific responses.
>>>
>>> On 2/20/2019 8:05 AM, Mirja Kühlewind wrote:
>>>> Mirja Kühlewind has entered the following ballot position for
>>>> draft-ietf-detnet-architecture-11: Discuss
>>>>
>>>> When responding, please keep the subject line intact and reply to all
>>>> email addresses included in the To and CC lines. (Feel free to cut 
>>>> this
>>>> introductory paragraph, however.)
>>>>
>>>>
>>>> Please refer to 
>>>> https://www.ietf.org/iesg/statement/discuss-criteria.html
>>>> for more information about IESG DISCUSS and COMMENT positions.
>>>>
>>>>
>>>> The document, along with other ballot positions, can be found here:
>>>> https://datatracker.ietf.org/doc/draft-ietf-detnet-architecture/
>>>>
>>>>
>>>>
>>>> ----------------------------------------------------------------------
>>>> DISCUSS:
>>>> ----------------------------------------------------------------------
>>>>
>>>> Thanks for addressing the tsv-art review comments  (and big thanks 
>>>> to Michael
>>>> and David!) and all the work done so far! I think the document is 
>>>> in good shape
>>>> and I only have one minor comment that I would like to see 
>>>> addressed/more
>>>> explicitly spelled out. However, this should be done quickly with 
>>>> potentially
>>>> 2-3 small changes in the draft. See below.
>>> The input from Michael and David certainly led to a much improved 
>>> document.  If you can help the authors with some specific text 
>>> suggestions that would be most helpful.
>>>
>>>
>>>> Given that DetNet traffic is often assumed to be not congestion 
>>>> controlled, it
>>>> is important that there is also some network function that makes 
>>>> sure the
>>>> source traffic stays within the requested bandwidth limit in-order 
>>>> to protect
>>>> non-Detnet traffic. This is to some extended discussed in section 
>>>> 3.3.2 but I
>>>> think it should be more clearly spelled out that this would require 
>>>> a rate
>>>> limiting function at each DetNet source/relay (tunnel ingress). 
>>>> Currently sec
>>>> 3.3.2 says:
>>>>
>>>> "Filters and policers should be used in a DetNet network to
>>>>     detect if DetNet packets are received on the wrong interface, 
>>>> or at
>>>>     the wrong time, or in too great a volume."
>>>>
>>>> However, maybe this case of limiting non-congestion controlled 
>>>> traffic (in case
>>>> the source in not keeping to the limits on purpose/in order to 
>>>> cheat, because
>>>> it couldn't estimate the needed bandwidth requirement an better, or 
>>>> due to
>>>> timely fluctuations) could be explained more clearing and the 
>>>> respective
>>>> requirement to implement rate limiting could be state separately 
>>>> and more
>>>> strongly...?
>>> Section 3.2.1.1 days:
>>>
>>>    Ensuring adequate buffering requires, in turn, that the source, and
>>>    every DetNet node along the path to the destination (or nearly every
>>>    node, see Section 4.3.3) be careful to regulate its output to not
>>>    exceed the data rate for any DetNet flow, except for brief periods
>>>    when making up for interfering traffic.  Any packet sent ahead of 
>>> its
>>>    time potentially adds to the number of buffers required by the next
>>>    hop DetNet node and may thus exceed the resources allocated for a
>>>    particular DetNet flow.
>>>
>>> Section 3.3.1 also says:
>>>
>>>    Starvation of non-DetNet traffic must be avoided, e.g., by traffic
>>>    policing functions (e.g., [RFC2475]).  Thus, the net effect of the
>>>    presence of DetNet flows in a network on the non-DetNet flows is
>>>    primarily a reduction in the available bandwidth.
>>>
>>> Is this not sufficient?  If not, can you suggest some additional 
>>> language?
>> My question is mostly if you only need to protect _complete_ 
>> starvation or any over-use by DetNet flows and how you would do that. 
>> If the answer is to rate limit the DetNet flows at ingress to the 
>> allocated bandwidth, that would completely address my concern. 
>> However note that buffer dimensioning and rate limiting is not the 
>> same. Rate limiting is usually a scheduling question, e.g. using a 
>> token bucket scheme or a fixed non-work conserving scheduler. 
>> Therefore it would be nice to add one additional sentence in section 
>> 3.2 that spells that out directly. e.g. something like:
>>
>> „In order to protect non-DetNet traffic from potentially misbehaving 
>> DetNet-Traffic sources, rate limiting, e.g. using a token bucket 
>> scheme, at each tunnel ingress must be applied.“
>
>> I’m sure you can adapt the wording to make it more DetNet-like 
>> language and maybe you also want to add another sentence explaining 
>> the problem, please feel free to do so but that’s more or less what 
>> I’m looking for.
>
> I do agree that we seem to be having a terminology issue here - and 
> that we are on the same page with what should happen.  The terminology 
> I've used/seen in past RFCs has usually used the terms 'shaping' and 
> 'policing' - and these are what we've been using within the WG.  For 
> example, both of these terms are used in RFC2475 but 'rate limiting' 
> is not. Would it work for you for the draft to use/adapt the rest of 
> your language, but keep 'by traffic policing functions (e.g., 
> [RFC2475])' rather than 'rate limiting, e.g. using a token bucket 
> scheme,' ?

We could extend the 2nd paragraph in 3.2.1.1 
(https://tools.ietf.org/html/draft-ietf-detnet-architecture-11#section-3.2.1.1) 
to capture this comment:

OLD:
             Ensuring adequate buffering requires, in turn, that the 
source, and every DetNet
             node along the path to the destination (or nearly every 
node, see
             <xref target="Incomplete"/>) be careful to regulate its 
output to not exceed the
             data rate for any DetNet flow, except for brief periods 
when making up for
             interfering traffic.  Any packet sent ahead of its time 
potentially adds to the
             number of buffers required by the next hop DetNet node and 
may thus exceed the resources
             allocated for a particular DetNet flow.

NEW:

             Ensuring adequate buffering requires, in turn, that the 
source, and every DetNet
             node along the path to the destination (or nearly every 
node, see
             <xref target="Incomplete"/>) be careful to regulate its 
output to not exceed the
             data rate for any DetNet flow, except for brief periods 
when making up for
             interfering traffic.  Any packet sent ahead of its time 
potentially adds to the
             number of buffers required by the next hop DetNet node and 
may thus exceed the resources
             allocated for a particular DetNet flow. Furthermore, rate 
limiting, e.g., using traffic
             policing and shaping functions, e.g., <xref 
target="RFC2475"/>, at the ingress of the
             DetNet domain must be applied. This is needed for meeting 
the requirements of DetNet
             flows as well as for protecting non-DetNet traffic from 
potentially misbehaving DetNet
             traffic sources.


Furthermore, replace the two occurrences of "policing" with "policing 
and shaping":

OLD:
    Starvation of non-DetNet traffic must be avoided, e.g., by traffic
    policing functions (e.g., [RFC2475]).

NEW:
    Starvation of non-DetNet traffic must be avoided, e.g., by traffic
    policing and shaping functions (e.g., [RFC2475]).


OLD:
    Examples of such techniques include traffic policing
    functions (e.g., [RFC2475]) and separating flows into per-flow rate-
    limited queues.

NEW:
    Examples of such techniques include traffic policing and shaping
    functions (e.g., [RFC2475]) and separating flows into per-flow rate-
    limited queues.

>
>
>>
>>>> One related comment on this sentence in Sec 3.1:
>>>>
>>>> "As DetNet provides allocated resources (including provisioned 
>>>> capacity)
>>>>     to DetNet flows the use of transport layer congestion control
>>>>     [RFC2914] by App-flows is explicitly not required."
>>>>
>>>> I guess congestion control should still be a requirement if the 
>>>> App-flow also
>>>> passes not-DetNet-aware segments of the path, e.g. maybe the first 
>>>> hop.
>>> This point is addressed in section 3.3.2
>>>
>>>    Note that the sending of App-flows that do not use transport layer
>>>    congestion control per [RFC2914] into a network that is not
>>>    provisioned to handle such DetNet traffic has to be treated as a
>>>    fault and prevented.
>>>
>>>> Usually
>>>> use of congestion control for application limited flows is also not 
>>>> a problem
>>>> if sufficient bandwidth is available. Also note that, as I stated 
>>>> above, the
>>>> important part for not requiring congestion control is actually not 
>>>> only that
>>>> resources are allocated but also that rate limiting is in place to 
>>>> make sure
>>>> resources usage cannot be exceeded above the reserved allocation. 
>>>> Maybe this
>>>> sentence could also be further clarified in the draft...?
>>> Is the above sufficient?  If not, can you suggest some additional 
>>> language?
>> Maybe we can just extend the above statement a bit in order to be 
>> crystal clear about this, e.g.:
>>
>> "As DetNet flow are assumed to be rate-limited and DetNet is designed 
>> to provide sufficient allocated resources (including provisioned 
>> capacity), the use of transport layer congestion control [RFC2914] 
>> for end-to-end DetNet flows is not required, however, if resources 
>> are allocated appropriately, use of congestion control should not 
>> impact transmission negatively.“
> Great!  (I'll leave it to the authors to do the final wording adaptation)

"App-flow" seems to be better than "end-to-end DetNet flow". The change 
with that would be:

OLD:
    As
    DetNet provides allocated resources (including provisioned capacity)
    to DetNet flows the use of transport layer congestion control
    [RFC2914] by App-flows is explicitly not required.

NEW:
             As
             DetNet flows are assumed to be rate-limited and DetNet is 
designed
             to provide sufficient allocated resources (including 
provisioned
             capacity), the use of transport layer congestion control 
[RFC2914]
             for App-flows is not required; however, if resources are 
allocated
             appropriately, use of congestion control should not impact
             transmission negatively.

>>
>>>> And then one more small comment that is also related. Sec 3.2.2.2 
>>>> says:
>>>>
>>>> "If packet replication and elimination is used over paths with
>>>>     resource allocation (Section 3.2.1), ..."
>>>>
>>>> My assumption was that all DetNet traffic is send over pre-allocated
>>>> resources...? If that is not true that has implication on 
>>>> congestion control
>>>> and needs some additional considerations. Can you please confirm 
>>>> that and maybe
>>>> clarify in the draft! Thanks!
>>> The intent was to cover sending over non-pre-allocated paths as a 
>>> fault case, and cover it in section 3.3.2.
>> If that is the case, maybe we can just removed the half-sentence I 
>> cited above?
>
> WFM!

The sentence after removing the text as you suggest:

OLD:
    If packet replication and elimination is used over paths with
    resource allocation (Section 3.2.1), and member flows that take
    different-length paths through the network are combined, a merge
    point may require extra buffering to equalize the delays over the
    different paths.

NEW:
    If member flows that take different-length paths through the
    network are combined, a merge point may require extra buffering
    to equalize the delays over the different paths.


Thank you!
Janos


>
> Thank you!
>
>>
>> Thanks!
>> Mirja
>>
>>> In addition to the previously quoted text, the section also says:
>>>
>>>    PRF generated DetNet member flows also need to
>>>    be treated as not using transport layer congestion control even if
>>>    the original App-flow supports transport layer congestion control
>>>    because PREOF can remove congestion indications at the PEF and
>>>    thereby hide such indications (e.g., drops, ECN markings, increased
>>>    latency) from end systems.
>>>
>>> Does this address your concern?
>>>> ----------------------------------------------------------------------
>>>> COMMENT:
>>>> ----------------------------------------------------------------------
>>>>
>>>> I agree with Alexey and Benjamin that this document should be 
>>>> informational.
>>>> Informational documents can also have IETF consensus, so that 
>>>> cannot be the
>>>> reason to go for PS. However, this document does not specify a 
>>>> protocol or any
>>>> requirements that are mandatory to implement for interoperability 
>>>> and therefore
>>>> should not be PS.
>>>>
>>> I defer to the IESG and AD on this...
>>>
>>> Thank you!
>>>
>>> Lou
>>>
>>>> _______________________________________________
>>>> detnet mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/detnet
>>>
>>