Re: [Tsv-art] [spring] TSV-ART review of draft-ietf-spring-segment-routing-msdc-08

[Gaurav Dawra <[e-mail]>]

Hi MIrja,

I have posted a new version. Please do let me know if we need to discuss
further. We can do it over the phone.

Cheers,

Gaurav


On Mon, Sep 3, 2018 at 4:29 AM Gaurav Dawra <[e-mail]> wrote:

> Hi Mirja,
>
> Please see inline...[Gaurav2]
>
> On Thu, Aug 9, 2018 at 8:30 AM Mirja Kühlewind <[e-mail]>
> wrote:
>
>> Hi Gaurav,
>>
>> please see inline.
>>
>> On 03.08.2018 07:20, Gaurav Dawra wrote:
>>
>> Hey Mirja,
>>
>> Sorry for the long delay. I was traveling constantly since IETF and bit
>> lost in my mailbox and discussion with Authors. Please see my response
>> inline[Gaurav]
>>
>>
>> I think with your changes you addressed explicit problems Martin called
>> out, however, I still have high level concerns about these sections as they
>> are mostly giving speculative recommendation which are not clear to me to
>> work in practice.
>>
>> Regarding section 7.1, you say
>> "A flowlet is defined as a burst of packets from the same flow followed
>> by an idle interval."
>> but then you say
>> "...then the application can break the elephant flow F into flowlets F1,
>> F2, F3, F4..."
>>
>> This sounds like you would just divide an elephant flow mostly randomly
>> into flowlets which can interact badly with the congestion control. If you
>> actually have chunks of data that are transmitted with large enough idle
>> interval in between (as you define flowlets in the first sentence), that is
>> not an elephant flow anymore and it will not help you to "spread the load
>> of the elephant flow through all the ECMP paths". In summary I actually
>> don't see how the concept of flowlets can be helpful to address the problem
>> you have at all, and I still advise you to remove section 7.1 entirely.
>>
>> [Gaurav] Hi Mirja, Thanks for the review. The proposal here is no
>> different that current ECMP hashing at flowlet level. The only difference
>> which is being pointed out here is that if we use SR, we could leverage on
>> the ability of be aware of multiple disjoint paths rather than the hashing.
>> It’s pins the flowlets to particular paths which is basic SR operations.
>>
>>
>> Yes the problem is that we usually don't recommend ECMP hashing on
>> flowlet level because it can interact badly with the end-end mechanisms of
>> that flow. As I said, I don't see how the notion of flowlets help you
>> problem and therefore I still recommend to remove that paragraph.
>> [Gaurav2] OK. It took sometime to get to consensus with authors. Will
>> update the text to use 5-tuple flows instead of flowlets. Would that
>> suffice? I will update the text shortly.
>>
>>
>> Regarding section 7.2, I also still skeptical about any benefits that can
>> be achieved. Given you are in a data center, the controller should already
>> know about static parameters such as the maximum bandwidth per link.
>>
>> For dynamic parameters, e.g. like loss rate, measuring them on a per-flow
>> bases is the wrong thing to do. What I mean is you can ask a router about
>> the average loss rate that it observes and that might give you some
>> valuable, however, if you ask a TCP flow about the average loss rate the
>> answer will mainly depend on the congestion controller used and the
>> currently available bandwidth, which is a very dynamic property and not a
>> link characteristic. So this information is not usable for performance
>> aware routing. A flow could give you information about the observed RTT
>> (min/max) on a certain path, or the maximum available bandwidth on a path,
>> but as I said, given you are in a data center environment these are
>> information that the controller already should have anyway.
>>
>> [Gaurav] They are two separate mechanisms. Most DCs have some sort of
>> data-plane/ECMP aware tracing mechanism to detect the loss/delays and can
>> be combined with Application back-off to detect issue. All this section is
>> suggesting is that SR can be used to pin the path to particular set of ECMP
>> paths instead of relying on ECMP hashing.
>>
>>
>> This is not quite what the text says. If that is the statement you want
>> to make, that is fine but then you don't need to talk about performance
>> aware routing at all.
>>
> [Gaurav2] I will update the text here with statement i mentioned above.
> IMHO, it's performance aware routing wrt to end-host traffic.
>
>>
>>
>> Your example with detecting a faulty path due to losses does not work
>> with TCP as you never know if these loses are due to a problem on the path,
>> self-induced or by a competing flow. And even if you don't use TCP and e.g.
>> send constant bit rate traffic, there may be a large number of competing
>> TCP flows that can induce the loses. Try to steer traffic "away" on a
>> time-scale that is slower than TCP dynamics or even your flow dynamic (when
>> flows start or end) in case you have a lot of very short flow, in the best
>> case doesn't work and in the worst case leads to oscillation.
>>
>> [Gaurav] As I said above, there are other mechanisms to detect loss and
>> trace the path on which loss is seen. This is a common mechanism used in
>> MSDCs.
>>
>> I think this is beyond the scope of the document.
>>
> [Gaurav2] Will update the text.
>
>>
>>
>>
>> I am happy to discuss further over the phone to try to explain the
>> thought process. I will also do check again with Authors to update the text
>> or something else based on our conversation.
>>
>>
>> Maybe see if some update can be made to the text first and then we can
>> have another discussion if needed.
>> [Gaurav2] Sounds good. Will update the text shortly and then we can
>> discuss if needed.
>>
>
> Cheers,
>
> Gaurav
>
>>
>>
>> Cheers,
>>
>>
>>
>> Gaurav
>>
>> If you want to make TCP use for handover situation where one path might
>> go away or become unusable, it's best to use Multipath TCP (with coupled
>> congestion control) instead because that works on the right time scale.
>> Again, I don't think this is a use case for SR and I would recommend to
>> remove the section entirely.
>>
>> Mirja
>>
>>
>>
>>
>> On Mon, Jul 16, 2018 at 11:08 PM, Gaurav Dawra <[e-mail]>
>> wrote:
>>
>>> Hi Mirja,
>>>
>>> Ack. Let me work with authors to close ASAP.
>>>
>>> Cheers,
>>>
>>> Gaurav
>>>
>>> Sent from my iPhone
>>>
>>> On Jul 5, 2018, at 10:06 AM, Mirja Kühlewind <[e-mail]>
>>> wrote:
>>>
>>> Hi Gaurav,
>>>
>>> sorry for my very long delay but this got somehow a bit lost in my
>>> mailbox.
>>>
>>> I think with your changes you addressed explicit problems Martin called
>>> out, however, I still have high level concerns about these sections as they
>>> are mostly giving speculative recommendation which are not clear to me to
>>> work in practice.
>>>
>>> Regarding section 7.1, you say
>>> "A flowlet is defined as a burst of packets from the same flow followed
>>> by an idle interval."
>>> but then you say
>>> "...then the application can break the elephant flow F into flowlets F1,
>>> F2, F3, F4..."
>>>
>>> This sounds like you would just divide an elephant flow mostly randomly
>>> into flowlets which can interact badly with the congestion control. If you
>>> actually have chunks of data that are transmitted with large enough idle
>>> interval in between (as you define flowlets in the first sentence), that is
>>> not an elephant flow anymore and it will not help you to "spread the load
>>> of the elephant flow through all the ECMP paths". In summary I actually
>>> don't see how the concept of flowlets can be helpful to address the problem
>>> you have at all, and I still advise you to remove section 7.1 entirely.
>>>
>>> Regarding section 7.2, I also still skeptical about any benefits that
>>> can be achieved. Given you are in a data center, the controller should
>>> already know about static parameters such as the maximum bandwidth per
>>> link. For dynamic parameters, e.g. like loss rate, measuring them on a
>>> per-flow bases is the wrong thing to do. What I mean is you can ask a
>>> router about the average loss rate that it observes and that might give you
>>> some valuable, however, if you ask a TCP flow about the average loss rate
>>> the answer will mainly depend on the congestion controller used and the
>>> currently available bandwidth, which is a very dynamic property and not a
>>> link characteristic. So this information is not usable for performance
>>> aware routing. A flow could give you information about the observed RTT
>>> (min/max) on a certain path, or the maximum available bandwidth on a path,
>>> but as I said, given you are in a data center environment these are
>>> information that the controller already should have anyway.
>>>
>>> Your example with detecting a faulty path due to losses does not work
>>> with TCP as you never know if these loses are due to a problem on the path,
>>> self-induced or by a competing flow. And even if you don't use TCP and e.g.
>>> send constant bit rate traffic, there may be a large number of competing
>>> TCP flows that can induce the loses. Try to steer traffic "away" on a
>>> time-scale that is slower than TCP dynamics or even your flow dynamic (when
>>> flows start or end) in case you have a lot of very short flow, in the best
>>> case doesn't work and in the worst case leads to oscillation.
>>>
>>> If you want to make TCP use for handover situation where one path might
>>> go away or become unusable, it's best to use Multipath TCP (with coupled
>>> congestion control) instead because that works on the right time scale.
>>> Again, I don't think this is a use case for SR and I would recommend to
>>> remove the section entirely.
>>>
>>> Mirja
>>>
>>>
>>> On 05.07.2018 04:08, Gaurav Dawra wrote:
>>>
>>> Hey Alvaro, Mirja,
>>>
>>> Friendly reminder to further progress this document.
>>>
>>> Cheers,
>>>
>>> Gaurav
>>>
>>> On Mon, Jun 18, 2018 at 5:13 PM, Gaurav Dawra <[e-mail]>
>>> wrote:
>>>
>>>> Hi Alvaro, Mirja
>>>>
>>>> Any feedback or next steps to close this?
>>>>
>>>> Cheers,
>>>>
>>>> Gaurav
>>>>
>>>> Sent from my iPhone
>>>>
>>>> On Jun 12, 2018, at 7:06 AM, Gaurav Dawra <[e-mail]>
>>>> wrote:
>>>>
>>>> Hi Mirja,
>>>>
>>>> Friendly Reminder...Could you please also advice if there is anything
>>>> further to DISCUSS on this document which was also related to TCP updates
>>>> below?
>>>>
>>>> Cheers,
>>>>
>>>> Gaurav
>>>>
>>>> On Thu, Jun 7, 2018 at 9:02 AM, Alvaro Retana <[e-mail]>
>>>> wrote:
>>>>
>>>>> Thanks Martin!
>>>>>
>>>>> On June 6, 2018 at 3:14:45 PM, Martin Stiemerling ([e-mail])
>>>>> wrote:
>>>>>
>>>>> Hi Alvaro, all,
>>>>>
>>>>> Thanks for addressing my concerns.
>>>>>
>>>>> This version is good to go from my side.
>>>>>
>>>>> Kind regards,
>>>>>
>>>>> ;Martin
>>>>>
>>>>> Am 30.05.18 um 21:55 schrieb Alvaro Retana:
>>>>> > Martin:
>>>>> > br/>> Hi!!  How are you?
>>>>> > br/>> Gaurav just posted a revision.  Please takke a look and let us
>>>>> know if br/>> the changes address your concerrns or not.
>>>>> > br/>>
>>>>> https://www.ietf.org/rfcdiff??url2=draft-ietf-spring-segment-routing-msdc-09
>>>>> > br/>> Thanks!!!
>>>>> > br/>> Alvaro. <
>>>>> > br/>> On May 25, 2018 at 12:08:46 PM, Gaurav Dawra ((
>>>>> [e-mail] br/>> <mailto:gdawra.ietf@@gmail.com>) wrote:
>>>>> > br/>>> Hi Martin, <
>>>>> >>
>>>>> >> Thanks for review. I will post the new version. Hopefully, it will
>>>>> br/>>> address all your comments and we can close thhis review.
>>>>> >>
>>>>> >> Any updates on below response?
>>>>> >>
>>>>> >> Cheers,
>>>>> >>
>>>>> >> Gaurav
>>>>> >>
>>>>> >> Sent from my iPhone
>>>>> >>
>>>>> >> On May 23, 2018, at 4:17 AM, Gaurav Dawra <[e-mail]
>>>>> br/>>> <mailto:gdawra.ietf@@gmail.com>> wrote:
>>>>> >>
>>>>> >>> Hi Martin,
>>>>> >>>
>>>>> >>> Thanks for the review. Any further comments here ? I will post the
>>>>> br/>>>> new version soon. <
>>>>> >>>
>>>>> >>> Cheers,
>>>>> >>>
>>>>> >>> Gaurav
>>>>> >>>
>>>>> >>> Sent from my iPhone
>>>>> >>>
>>>>> >>> On May 16, 2018, at 7:44 PM, Gaurav Dawra <[e-mail]
>>>>> br/>>>> <mailto:gdawra.ietf@@gmail..com <http://gmail.com>>> wrote:
>>>>> >>>
>>>>> >>>> Hi Martin,
>>>>> >>>>
>>>>> >>>> Apologies from my end we had few internal authors conversations
>>>>> on br/>>>>> the points you have raised. <
>>>>> >>>>
>>>>> >>>> Please find below my response. I will be happy to discuss
>>>>> further, br/>>>>> if needed. <
>>>>> >>>>
>>>>> >>>> <Gaurav> inline...
>>>>> >>>>
>>>>> >>>>> On Apr 9, 2018, at 7:58 AM, Martin Stiemerling <
>>>>> [e-mail] br/>>>>>> <mailto:[e-mail]>> wrote:
>>>>> >>>>>
>>>>> >>>>> Hi Gaurav,
>>>>> >>>>>
>>>>> >>>>> This got lost on my end, sorry for this. The filter just moved
>>>>> br/>>>>>> these messages out of my sight... :-/
>>>>> >>>>>
>>>>> >>>>> Am 15.02.18 um 05:47 schrieb Gaurav Dawra:
>>>>> >>>>>> Hey Martin,
>>>>> >>>>>> Sorry for late reply. Please see some comments inline[Gaurav]
>>>>> >>>>>>> On Jan 9, 2018, at 2:25 PM, Martin Stiemerling br/>>>>>>>>
>>>>> <mls.ietf@@gmail.com <mailto:[e-mail]> br/>>>>>>>>; <mailto:
>>>>> [e-mail]>> wrote:
>>>>> >>>>>>>
>>>>> >>>>>>> Hi all,
>>>>> >>>>>>>
>>>>> >>>>>>> I've reviewed this document as part of the transport area
>>>>> review br/>>>>>>>> team's ongoing effort to review key IETF documents.
>>>>> These br/>>>&gtt;>>>> comments were written primarily for the transport
>>>>> area directors, br/>>>>>>>> but are copied to the doocument's authors for
>>>>> their information br/>>>>>>>&> and to allow them to address any issues
>>>>> raised. When done at the
>>>>> >>>>>>> time of IETF Last Call, the authors should consider this
>>>>> review br/>>>>>>>> together with any other last-call comments they receive.
>>>>> Please br/>>>&>>>>> always CC tsv-art@… if you reply to or forward
>>>>> this review.
>>>>> >>>>>>>
>>>>> >>>>>>> Summary:
>>>>> >>>>>>> This draft has serious issues in Section 7..1, 7.2 and in one
>>>>> part br/>>>>>>>> of Secction3, described in the review, and needs to be
>>>>> rethought. br/>>&>>>>>> The other sections are good AFAIK.
>>>>> >>>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>> Technicals:
>>>>> >>>>>>> The overall draft looks ok, but the three points below look
>>>>> br/>>>>>>>> strange and need a fix before publication IMHO:
>>>>> >>>>>>>
>>>>> >>>>>>> Both Sections, 7.1. and 7.2., are describing ideas, but not
>>>>> well br/>>>>>>>> proven funcationality and not even safe to use
>>>>> functionality. br/>>>&>>>>> Both are some sort discussing that different
>>>>> paths in the network br/>>>>>>>> could be used by the eend host traffic.
>>>>> This sounds pretty much br/>>>>>>&gtt;> like the Path Aware Networking
>>>>> Proposed Research Group br/>&gtt;>>>>>> (https://irtf.org/panrg) and
>>>>> hints to the fact that there is no br/>>>>>>>> commonly understannd and
>>>>> accepted engineering solution in this space.
>>>>> >>>>>>>
>>>>> >>>>>>> Section 7.1:
>>>>> >>>>>>> [KANDULA04] is a really old reference that hasn't been
>>>>> followed br/>>>>>>>> up iin recent times and even worse there is no
>>>>> evidence that this br/>&gtt;>>>>>> is going to work good enough or stable
>>>>> enough under real Internet br/>>>>>>>> traffic. Additioonally, it is more
>>>>> than unclear how any modern TCP br/>>>>&ggt;>>> implementation will react
>>>>> to this
>>>>> >>>>>> [Gaurav] Will get back on this.
>>>>> >>>>>
>>>>> >>>>> I will reply to the other email dicussing this.
>>>>> >>>> <Gaurav> I have replied to other thread.
>>>>> >>>>>
>>>>> >>>>>>>
>>>>> >>>>>>> Section 7.2:
>>>>> >>>>>>> This section describes an idea without detailing too much
>>>>> about br/>>>>>>>> any furtther aspects. Further it changes the commonly
>>>>> accepted br/>>>;>>>>> notion of what an end host can do with the network.
>>>>> At best this br/>>>>>>>> would require a good ddefinition of what an end
>>>>> host in your br/>>>>>>>&ggt; setting is, e.g., a highly modified piece of
>>>>> (at least) software
>>>>> >>>>>>> that usually not found in OS availble on the market (yet?)
>>>>> >>>>>>> Further communicating instantaneous path characteristics to a
>>>>> br/>>>>>>>> central point is potentially a bad idea, as the data is already
>>>>> br/>>>;>>>>> outdated when reported by any node.
>>>>> >>>>>> [Gaurav] I believe Authors are trying to highlight that Host
>>>>> which br/>>>>>>> is defineed in br/>>>>>>> (
>>>>> https://tools.ietf.org/html/draftt-ietf-spring-segment-routing-15)
>>>>> br/>>>>>>> can innfluence the traffic based on the Calculations locally or
>>>>> br/>>&gtt;>>>> jointly with the controller. Implementations can decide how
>>>>> much br/>>>>>>> Centralized -vs- localized coontrol is allowed at Host
>>>>> based on br/>>>>>>> perfoormance data collection.
>>>>> >>>>>
>>>>> >>>>> Performance data collection (monitoring?) isn't crucial when it
>>>>> br/>>>>>> comes to timely (actuaally real-time) reaction. However, this
>>>>> br/>>>>>> secttion isn't just about performance data collection as it is
>>>>> about br/>>>>>>> "Performance-aware routing" this seems to try to interact
>>>>> in br/>>>>>> real-time with the network behhavior of TCP. This isn't even
>>>>> close br/>>>>>> to acceeptable.
>>>>> >>>>>
>>>>> >>>>> I would be ok to say that it is useful to collect performance
>>>>> data br/>>>>>> for offline analysis and improvement of the data network.
>>>>> However, br/>>>>>&ggt; this is at completely different magnitues of time
>>>>> scales.
>>>>> >>>>>
>>>>> >>>>> I would recommend to remove the TCP part from this section
>>>>> entirely.
>>>>> >>>> <Gaurav>Ack, will update in next rev:
>>>>> >>>>
>>>>> >>>> Section will read like this:
>>>>> >>>>
>>>>> >>>> ;
>>>>> >>>> /Knowing the path associated with flows/packets, the end host
>>>>> may/
>>>>> >>>> /deduce certain characteristics of the path on its own, and/
>>>>> >>>> /additionally use the information supplied with path information/
>>>>> >>>> /pushed from the controller or received via pull request. The
>>>>> host/
>>>>> >>>> /may further share its path observations with the centralized
>>>>> agent,/
>>>>> >>>> /so that the latter may keep up-to-date network health map to
>>>>> assist/
>>>>> >>>> /other hosts with this information./
>>>>> >>>> //
>>>>> >>>> /For example, an application A.1 at HostA may pin a flow
>>>>> destined/
>>>>> >>>> /to HostZ via Spine node Node5 using label stack {16005, 16011}.
>>>>> The/
>>>>> >>>> /application A.1 may collect information on packet loss, deduced
>>>>> from/
>>>>> >>>> /Other offline mechanisms. [There are some pingMesh mechanisms
>>>>> which /
>>>>> >>>> /Can be used here]/
>>>>> >>>> /Through these mechanisms information to a centralized agent,
>>>>> e.g./
>>>>> >>>> /after a flow completes, or periodically for longer lived flows./
>>>>> >>>> /Next, using both local and/or global performance data,
>>>>> application/
>>>>> >>>> /A.1 as well as other applications sharing the same resources in
>>>>> the/
>>>>> >>>> /DC fabric may pick up the best path for the new flow, or update
>>>>> an/
>>>>> >>>> /existing path (e.g.: when informed of congestion on an existing/
>>>>> >>>> /path)./
>>>>> >>>> ;
>>>>> >>>>
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>>>>
>>>>> >>>>>>> Section 3, 3rd bullet point:
>>>>> >>>>>>> It is the foundation of TCP that the network is regarded as a
>>>>> br/>>>>>>>> black box and that you infer from the transmission of packets
>>>>> br/>>>>;>>>> what the current state of the network path is. Inferring
>>>>> network br/>>>>>>>> path metrics (you mention SRTT, MSS, CWND ) is a bad
>>>>> idea, as br/>>>>>>>>; this would required that all paths exhibit this and
>>>>> if not what br//>>>>>>>> is going to happen?
>>>>> >>>>>>> It could be an interesting research field to change many
>>>>> points br/>>>>>>>> in TCP'ss behavior, but this once again points to the
>>>>> fact that br/>>>&>>>>> this not the IETF works but IRTF or elsewhere.
>>>>> >>>>>> [Gaurav] Martin, Authors are trying to suggest that TCP is
>>>>> rightly br/>>>>>>> treating Network as Black Box. Authors are implying per
>>>>> path br/>>>>;>>> performance metrics as not cached. Is there some change in
>>>>> text br/>>>>>>> you are suggesting??
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> I would recommend to remove the 3rd bullet point completey. TCP
>>>>> br/>>>>>> isn't the place to rememmber "good" or "bad" paths. This is
>>>>> br/>>>>>> something the network could decide, e.g., rerouting TCP flows
>>>>> br/>&ggt;>>>> within the network or changing the forwarding path in the
>>>>> network br/>>>>>> for particular flows (if it is not routed).
>>>>> >>>>
>>>>> >>>> <Gaurav> Ack, after discussion, we will remove the Section 3 -
>>>>> 3rd br/>>>>> bullet point. Willl update in next rev - coming shortly.
>>>>> >>>>
>>>>> >>>>>
>>>>> >>>>> Kind regards,
>>>>> >>>>>
>>>>> >>>>>  Martin
>>>>> >>>>>
>>>>> >>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>> _______________________________________________
>>> Tsv-art mailing [e-mail]://www.ietf.org/mailman/listinfo/tsv-art
>>>
>>>
>>>
>>
>>