Re: [spring] [Tsv-art] TSV-ART review of draft-ietf-spring-segment-routing-msdc-08

[Gaurav Dawra <[e-mail]>]

Hi Mirja,

Please see inline...[Gaurav2]

On Thu, Aug 9, 2018 at 8:30 AM Mirja Kühlewind <[e-mail]> wrote:

> Hi Gaurav,
>
> please see inline.
>
> On 03.08.2018 07:20, Gaurav Dawra wrote:
>
> Hey Mirja,
>
> Sorry for the long delay. I was traveling constantly since IETF and bit
> lost in my mailbox and discussion with Authors. Please see my response
> inline[Gaurav]
>
>
> I think with your changes you addressed explicit problems Martin called
> out, however, I still have high level concerns about these sections as they
> are mostly giving speculative recommendation which are not clear to me to
> work in practice.
>
> Regarding section 7.1, you say
> "A flowlet is defined as a burst of packets from the same flow followed by
> an idle interval."
> but then you say
> "...then the application can break the elephant flow F into flowlets F1,
> F2, F3, F4..."
>
> This sounds like you would just divide an elephant flow mostly randomly
> into flowlets which can interact badly with the congestion control. If you
> actually have chunks of data that are transmitted with large enough idle
> interval in between (as you define flowlets in the first sentence), that is
> not an elephant flow anymore and it will not help you to "spread the load
> of the elephant flow through all the ECMP paths". In summary I actually
> don't see how the concept of flowlets can be helpful to address the problem
> you have at all, and I still advise you to remove section 7.1 entirely.
>
> [Gaurav] Hi Mirja, Thanks for the review. The proposal here is no
> different that current ECMP hashing at flowlet level. The only difference
> which is being pointed out here is that if we use SR, we could leverage on
> the ability of be aware of multiple disjoint paths rather than the hashing.
> It’s pins the flowlets to particular paths which is basic SR operations.
>
>
> Yes the problem is that we usually don't recommend ECMP hashing on flowlet
> level because it can interact badly with the end-end mechanisms of that
> flow. As I said, I don't see how the notion of flowlets help you problem
> and therefore I still recommend to remove that paragraph.
> [Gaurav2] OK. It took sometime to get to consensus with authors. Will
> update the text to use 5-tuple flows instead of flowlets. Would that
> suffice? I will update the text shortly.
>
>
> Regarding section 7.2, I also still skeptical about any benefits that can
> be achieved. Given you are in a data center, the controller should already
> know about static parameters such as the maximum bandwidth per link.
>
> For dynamic parameters, e.g. like loss rate, measuring them on a per-flow
> bases is the wrong thing to do. What I mean is you can ask a router about
> the average loss rate that it observes and that might give you some
> valuable, however, if you ask a TCP flow about the average loss rate the
> answer will mainly depend on the congestion controller used and the
> currently available bandwidth, which is a very dynamic property and not a
> link characteristic. So this information is not usable for performance
> aware routing. A flow could give you information about the observed RTT
> (min/max) on a certain path, or the maximum available bandwidth on a path,
> but as I said, given you are in a data center environment these are
> information that the controller already should have anyway.
>
> [Gaurav] They are two separate mechanisms. Most DCs have some sort of
> data-plane/ECMP aware tracing mechanism to detect the loss/delays and can
> be combined with Application back-off to detect issue. All this section is
> suggesting is that SR can be used to pin the path to particular set of ECMP
> paths instead of relying on ECMP hashing.
>
>
> This is not quite what the text says. If that is the statement you want to
> make, that is fine but then you don't need to talk about performance aware
> routing at all.
>
[Gaurav2] I will update the text here with statement i mentioned above.
IMHO, it's performance aware routing wrt to end-host traffic.

>
>
> Your example with detecting a faulty path due to losses does not work with
> TCP as you never know if these loses are due to a problem on the path,
> self-induced or by a competing flow. And even if you don't use TCP and e.g.
> send constant bit rate traffic, there may be a large number of competing
> TCP flows that can induce the loses. Try to steer traffic "away" on a
> time-scale that is slower than TCP dynamics or even your flow dynamic (when
> flows start or end) in case you have a lot of very short flow, in the best
> case doesn't work and in the worst case leads to oscillation.
>
> [Gaurav] As I said above, there are other mechanisms to detect loss and
> trace the path on which loss is seen. This is a common mechanism used in
> MSDCs.
>
> I think this is beyond the scope of the document.
>
[Gaurav2] Will update the text.

>
>
>
> I am happy to discuss further over the phone to try to explain the thought
> process. I will also do check again with Authors to update the text or
> something else based on our conversation.
>
>
> Maybe see if some update can be made to the text first and then we can
> have another discussion if needed.
> [Gaurav2] Sounds good. Will update the text shortly and then we can
> discuss if needed.
>

Cheers,

Gaurav

>
>
> Cheers,
>
>
>
> Gaurav
>
> If you want to make TCP use for handover situation where one path might go
> away or become unusable, it's best to use Multipath TCP (with coupled
> congestion control) instead because that works on the right time scale.
> Again, I don't think this is a use case for SR and I would recommend to
> remove the section entirely.
>
> Mirja
>
>
>
>
> On Mon, Jul 16, 2018 at 11:08 PM, Gaurav Dawra <[e-mail]>
> wrote:
>
>> Hi Mirja,
>>
>> Ack. Let me work with authors to close ASAP.
>>
>> Cheers,
>>
>> Gaurav
>>
>> Sent from my iPhone
>>
>> On Jul 5, 2018, at 10:06 AM, Mirja Kühlewind <[e-mail]> wrote:
>>
>> Hi Gaurav,
>>
>> sorry for my very long delay but this got somehow a bit lost in my
>> mailbox.
>>
>> I think with your changes you addressed explicit problems Martin called
>> out, however, I still have high level concerns about these sections as they
>> are mostly giving speculative recommendation which are not clear to me to
>> work in practice.
>>
>> Regarding section 7.1, you say
>> "A flowlet is defined as a burst of packets from the same flow followed
>> by an idle interval."
>> but then you say
>> "...then the application can break the elephant flow F into flowlets F1,
>> F2, F3, F4..."
>>
>> This sounds like you would just divide an elephant flow mostly randomly
>> into flowlets which can interact badly with the congestion control. If you
>> actually have chunks of data that are transmitted with large enough idle
>> interval in between (as you define flowlets in the first sentence), that is
>> not an elephant flow anymore and it will not help you to "spread the load
>> of the elephant flow through all the ECMP paths". In summary I actually
>> don't see how the concept of flowlets can be helpful to address the problem
>> you have at all, and I still advise you to remove section 7.1 entirely.
>>
>> Regarding section 7.2, I also still skeptical about any benefits that can
>> be achieved. Given you are in a data center, the controller should already
>> know about static parameters such as the maximum bandwidth per link. For
>> dynamic parameters, e.g. like loss rate, measuring them on a per-flow bases
>> is the wrong thing to do. What I mean is you can ask a router about the
>> average loss rate that it observes and that might give you some valuable,
>> however, if you ask a TCP flow about the average loss rate the answer will
>> mainly depend on the congestion controller used and the currently available
>> bandwidth, which is a very dynamic property and not a link characteristic.
>> So this information is not usable for performance aware routing. A flow
>> could give you information about the observed RTT (min/max) on a certain
>> path, or the maximum available bandwidth on a path, but as I said, given
>> you are in a data center environment these are information that the
>> controller already should have anyway.
>>
>> Your example with detecting a faulty path due to losses does not work
>> with TCP as you never know if these loses are due to a problem on the path,
>> self-induced or by a competing flow. And even if you don't use TCP and e.g.
>> send constant bit rate traffic, there may be a large number of competing
>> TCP flows that can induce the loses. Try to steer traffic "away" on a
>> time-scale that is slower than TCP dynamics or even your flow dynamic (when
>> flows start or end) in case you have a lot of very short flow, in the best
>> case doesn't work and in the worst case leads to oscillation.
>>
>> If you want to make TCP use for handover situation where one path might
>> go away or become unusable, it's best to use Multipath TCP (with coupled
>> congestion control) instead because that works on the right time scale.
>> Again, I don't think this is a use case for SR and I would recommend to
>> remove the section entirely.
>>
>> Mirja
>>
>>
>> On 05.07.2018 04:08, Gaurav Dawra wrote:
>>
>> Hey Alvaro, Mirja,
>>
>> Friendly reminder to further progress this document.
>>
>> Cheers,
>>
>> Gaurav
>>
>> On Mon, Jun 18, 2018 at 5:13 PM, Gaurav Dawra <[e-mail]>
>> wrote:
>>
>>> Hi Alvaro, Mirja
>>>
>>> Any feedback or next steps to close this?
>>>
>>> Cheers,
>>>
>>> Gaurav
>>>
>>> Sent from my iPhone
>>>
>>> On Jun 12, 2018, at 7:06 AM, Gaurav Dawra <[e-mail]> wrote:
>>>
>>> Hi Mirja,
>>>
>>> Friendly Reminder...Could you please also advice if there is anything
>>> further to DISCUSS on this document which was also related to TCP updates
>>> below?
>>>
>>> Cheers,
>>>
>>> Gaurav
>>>
>>> On Thu, Jun 7, 2018 at 9:02 AM, Alvaro Retana <[e-mail]>
>>> wrote:
>>>
>>>> Thanks Martin!
>>>>
>>>> On June 6, 2018 at 3:14:45 PM, Martin Stiemerling ([e-mail])
>>>> wrote:
>>>>
>>>> Hi Alvaro, all,
>>>>
>>>> Thanks for addressing my concerns.
>>>>
>>>> This version is good to go from my side.
>>>>
>>>> Kind regards,
>>>>
>>>> ;Martin
>>>>
>>>> Am 30.05.18 um 21:55 schrieb Alvaro Retana:
>>>> > Martin:
>>>> > br/>> Hi!!  How are you?
>>>> > br/>> Gaurav just posted a revision.  Please takke a look and let us
>>>> know if br/>> the changes address your concerrns or not.
>>>> > br/>>
>>>> https://www.ietf.org/rfcdiff??url2=draft-ietf-spring-segment-routing-msdc-09
>>>> > br/>> Thanks!!!
>>>> > br/>> Alvaro. <
>>>> > br/>> On May 25, 2018 at 12:08:46 PM, Gaurav Dawra ((
>>>> [e-mail] br/>> <mailto:gdawra.ietf@@gmail.com>) wrote:
>>>> > br/>>> Hi Martin, <
>>>> >>
>>>> >> Thanks for review. I will post the new version. Hopefully, it will
>>>> br/>>> address all your comments and we can close thhis review.
>>>> >>
>>>> >> Any updates on below response?
>>>> >>
>>>> >> Cheers,
>>>> >>
>>>> >> Gaurav
>>>> >>
>>>> >> Sent from my iPhone
>>>> >>
>>>> >> On May 23, 2018, at 4:17 AM, Gaurav Dawra <[e-mail]
>>>> br/>>> <mailto:gdawra.ietf@@gmail.com>> wrote:
>>>> >>
>>>> >>> Hi Martin,
>>>> >>>
>>>> >>> Thanks for the review. Any further comments here ? I will post the
>>>> br/>>>> new version soon. <
>>>> >>>
>>>> >>> Cheers,
>>>> >>>
>>>> >>> Gaurav
>>>> >>>
>>>> >>> Sent from my iPhone
>>>> >>>
>>>> >>> On May 16, 2018, at 7:44 PM, Gaurav Dawra <[e-mail]
>>>> br/>>>> <mailto:gdawra.ietf@@gmail..com <http://gmail.com>>> wrote:
>>>> >>>
>>>> >>>> Hi Martin,
>>>> >>>>
>>>> >>>> Apologies from my end we had few internal authors conversations on
>>>> br/>>>>> the points you have raised. <
>>>> >>>>
>>>> >>>> Please find below my response. I will be happy to discuss further,
>>>> br/>>>>> if needed. <
>>>> >>>>
>>>> >>>> <Gaurav> inline...
>>>> >>>>
>>>> >>>>> On Apr 9, 2018, at 7:58 AM, Martin Stiemerling <
>>>> [e-mail] br/>>>>>> <mailto:[e-mail]>> wrote:
>>>> >>>>>
>>>> >>>>> Hi Gaurav,
>>>> >>>>>
>>>> >>>>> This got lost on my end, sorry for this. The filter just moved
>>>> br/>>>>>> these messages out of my sight... :-/
>>>> >>>>>
>>>> >>>>> Am 15.02.18 um 05:47 schrieb Gaurav Dawra:
>>>> >>>>>> Hey Martin,
>>>> >>>>>> Sorry for late reply. Please see some comments inline[Gaurav]
>>>> >>>>>>> On Jan 9, 2018, at 2:25 PM, Martin Stiemerling br/>>>>>>>>
>>>> <mls.ietf@@gmail.com <mailto:[e-mail]> br/>>>>>>>>; <mailto:
>>>> [e-mail]>> wrote:
>>>> >>>>>>>
>>>> >>>>>>> Hi all,
>>>> >>>>>>>
>>>> >>>>>>> I've reviewed this document as part of the transport area
>>>> review br/>>>>>>>> team's ongoing effort to review key IETF documents.
>>>> These br/>>>&gtt;>>>> comments were written primarily for the transport
>>>> area directors, br/>>>>>>>> but are copied to the doocument's authors for
>>>> their information br/>>>>>>>&> and to allow them to address any issues
>>>> raised. When done at the
>>>> >>>>>>> time of IETF Last Call, the authors should consider this review
>>>> br/>>>>>>>> together with any other last-call comments they receive. Please
>>>> br/>>>&>>>>> always CC tsv-art@… if you reply to or forward this
>>>> review.
>>>> >>>>>>>
>>>> >>>>>>> Summary:
>>>> >>>>>>> This draft has serious issues in Section 7..1, 7.2 and in one
>>>> part br/>>>>>>>> of Secction3, described in the review, and needs to be
>>>> rethought. br/>>&>>>>>> The other sections are good AFAIK.
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>> Technicals:
>>>> >>>>>>> The overall draft looks ok, but the three points below look
>>>> br/>>>>>>>> strange and need a fix before publication IMHO:
>>>> >>>>>>>
>>>> >>>>>>> Both Sections, 7.1. and 7.2., are describing ideas, but not
>>>> well br/>>>>>>>> proven funcationality and not even safe to use
>>>> functionality. br/>>>&>>>>> Both are some sort discussing that different
>>>> paths in the network br/>>>>>>>> could be used by the eend host traffic.
>>>> This sounds pretty much br/>>>>>>&gtt;> like the Path Aware Networking
>>>> Proposed Research Group br/>&gtt;>>>>>> (https://irtf.org/panrg) and
>>>> hints to the fact that there is no br/>>>>>>>> commonly understannd and
>>>> accepted engineering solution in this space.
>>>> >>>>>>>
>>>> >>>>>>> Section 7.1:
>>>> >>>>>>> [KANDULA04] is a really old reference that hasn't been followed
>>>> br/>>>>>>>> up iin recent times and even worse there is no evidence that
>>>> this br/>&gtt;>>>>>> is going to work good enough or stable enough under
>>>> real Internet br/>>>>>>>> traffic. Additioonally, it is more than unclear
>>>> how any modern TCP br/>>>>&ggt;>>> implementation will react to this
>>>> >>>>>> [Gaurav] Will get back on this.
>>>> >>>>>
>>>> >>>>> I will reply to the other email dicussing this.
>>>> >>>> <Gaurav> I have replied to other thread.
>>>> >>>>>
>>>> >>>>>>>
>>>> >>>>>>> Section 7.2:
>>>> >>>>>>> This section describes an idea without detailing too much about
>>>> br/>>>>>>>> any furtther aspects. Further it changes the commonly accepted
>>>> br/>>>;>>>>> notion of what an end host can do with the network. At best
>>>> this br/>>>>>>>> would require a good ddefinition of what an end host in
>>>> your br/>>>>>>>&ggt; setting is, e.g., a highly modified piece of (at
>>>> least) software
>>>> >>>>>>> that usually not found in OS availble on the market (yet?)
>>>> >>>>>>> Further communicating instantaneous path characteristics to a
>>>> br/>>>>>>>> central point is potentially a bad idea, as the data is already
>>>> br/>>>;>>>>> outdated when reported by any node.
>>>> >>>>>> [Gaurav] I believe Authors are trying to highlight that Host
>>>> which br/>>>>>>> is defineed in br/>>>>>>> (
>>>> https://tools.ietf.org/html/draftt-ietf-spring-segment-routing-15)
>>>> br/>>>>>>> can innfluence the traffic based on the Calculations locally or
>>>> br/>>&gtt;>>>> jointly with the controller. Implementations can decide how
>>>> much br/>>>>>>> Centralized -vs- localized coontrol is allowed at Host
>>>> based on br/>>>>>>> perfoormance data collection.
>>>> >>>>>
>>>> >>>>> Performance data collection (monitoring?) isn't crucial when it
>>>> br/>>>>>> comes to timely (actuaally real-time) reaction. However, this
>>>> br/>>>>>> secttion isn't just about performance data collection as it is
>>>> about br/>>>>>>> "Performance-aware routing" this seems to try to interact
>>>> in br/>>>>>> real-time with the network behhavior of TCP. This isn't even
>>>> close br/>>>>>> to acceeptable.
>>>> >>>>>
>>>> >>>>> I would be ok to say that it is useful to collect performance
>>>> data br/>>>>>> for offline analysis and improvement of the data network.
>>>> However, br/>>>>>&ggt; this is at completely different magnitues of time
>>>> scales.
>>>> >>>>>
>>>> >>>>> I would recommend to remove the TCP part from this section
>>>> entirely.
>>>> >>>> <Gaurav>Ack, will update in next rev:
>>>> >>>>
>>>> >>>> Section will read like this:
>>>> >>>>
>>>> >>>> ;
>>>> >>>> /Knowing the path associated with flows/packets, the end host may/
>>>> >>>> /deduce certain characteristics of the path on its own, and/
>>>> >>>> /additionally use the information supplied with path information/
>>>> >>>> /pushed from the controller or received via pull request. The
>>>> host/
>>>> >>>> /may further share its path observations with the centralized
>>>> agent,/
>>>> >>>> /so that the latter may keep up-to-date network health map to
>>>> assist/
>>>> >>>> /other hosts with this information./
>>>> >>>> //
>>>> >>>> /For example, an application A.1 at HostA may pin a flow destined/
>>>> >>>> /to HostZ via Spine node Node5 using label stack {16005, 16011}.
>>>> The/
>>>> >>>> /application A.1 may collect information on packet loss, deduced
>>>> from/
>>>> >>>> /Other offline mechanisms. [There are some pingMesh mechanisms
>>>> which /
>>>> >>>> /Can be used here]/
>>>> >>>> /Through these mechanisms information to a centralized agent,
>>>> e.g./
>>>> >>>> /after a flow completes, or periodically for longer lived flows./
>>>> >>>> /Next, using both local and/or global performance data,
>>>> application/
>>>> >>>> /A.1 as well as other applications sharing the same resources in
>>>> the/
>>>> >>>> /DC fabric may pick up the best path for the new flow, or update
>>>> an/
>>>> >>>> /existing path (e.g.: when informed of congestion on an existing/
>>>> >>>> /path)./
>>>> >>>> ;
>>>> >>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>>>
>>>> >>>>>>> Section 3, 3rd bullet point:
>>>> >>>>>>> It is the foundation of TCP that the network is regarded as a
>>>> br/>>>>>>>> black box and that you infer from the transmission of packets
>>>> br/>>>>;>>>> what the current state of the network path is. Inferring
>>>> network br/>>>>>>>> path metrics (you mention SRTT, MSS, CWND ) is a bad
>>>> idea, as br/>>>>>>>>; this would required that all paths exhibit this and
>>>> if not what br//>>>>>>>> is going to happen?
>>>> >>>>>>> It could be an interesting research field to change many points
>>>> br/>>>>>>>> in TCP'ss behavior, but this once again points to the fact that
>>>> br/>>>&>>>>> this not the IETF works but IRTF or elsewhere.
>>>> >>>>>> [Gaurav] Martin, Authors are trying to suggest that TCP is
>>>> rightly br/>>>>>>> treating Network as Black Box. Authors are implying per
>>>> path br/>>>>;>>> performance metrics as not cached. Is there some change in
>>>> text br/>>>>>>> you are suggesting??
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> I would recommend to remove the 3rd bullet point completey. TCP
>>>> br/>>>>>> isn't the place to rememmber "good" or "bad" paths. This is
>>>> br/>>>>>> something the network could decide, e.g., rerouting TCP flows
>>>> br/>&ggt;>>>> within the network or changing the forwarding path in the
>>>> network br/>>>>>> for particular flows (if it is not routed).
>>>> >>>>
>>>> >>>> <Gaurav> Ack, after discussion, we will remove the Section 3 - 3rd
>>>> br/>>>>> bullet point. Willl update in next rev - coming shortly.
>>>> >>>>
>>>> >>>>>
>>>> >>>>> Kind regards,
>>>> >>>>>
>>>> >>>>>  Martin
>>>> >>>>>
>>>> >>>>
>>>>
>>>>
>>>
>>
>>
>> _______________________________________________
>> Tsv-art mailing [e-mail]://www.ietf.org/mailman/listinfo/tsv-art
>>
>>
>>
>
>