Re: [Lsr] New draft on Flex-Algorithm Bandwidth Constraints

Peter Psenak <ppsenak@cisco.com> Thu, 04 March 2021 11:29 UTC

Return-Path: <ppsenak@cisco.com>
X-Original-To: lsr@ietfa.amsl.com
Delivered-To: lsr@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3A3273A196E for <lsr@ietfa.amsl.com>; Thu, 4 Mar 2021 03:29:00 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.602
X-Spam-Level:
X-Spam-Status: No, score=-9.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zx_y6h_NxCgu for <lsr@ietfa.amsl.com>; Thu, 4 Mar 2021 03:28:57 -0800 (PST)
Received: from aer-iport-4.cisco.com (aer-iport-4.cisco.com [173.38.203.54]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5B7F53A196D for <lsr@ietf.org>; Thu, 4 Mar 2021 03:28:57 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=19286; q=dns/txt; s=iport; t=1614857337; x=1616066937; h=subject:to:cc:references:from:message-id:date: mime-version:in-reply-to:content-transfer-encoding; bh=QYpZUxINBctVIuvVpZM7MO2hCaW+/NnDhRYQQ6R1OvA=; b=BZ3EitokfKC07pAuKIfv2EBlhZrBH7XirSkxcuCUmMJhmztuSBtKSjlF A04NUlu/Y+fgMya4oVw6Y5j/NAoo5iSJ7wHg1ib9myIUJCSSRLS7dxKRc pt06oLU0kA7wskj0kAsM1QhdbrJn4VgjgDFd5paKIALbki7omEMEbE86j E=;
X-IPAS-Result: =?us-ascii?q?A0CaAQAkxEBglxbLJq1iGwEBAQEBAQEBBQEBARIBAQEDA?= =?us-ascii?q?wEBAUCBT4MhUgQBJxIxhEGJBIgoLQOcSwsBAQEPKAwEAQGETQKBeyY4EwIDA?= =?us-ascii?q?QEBAwIDAQEBAQUBAQECAQYEFAEBAQEBAQEBhjYNhkQBAQEDASMPAQVBEAkCG?= =?us-ascii?q?AICIwMCAkYRBg0GAgEBgmwBgmYhD5JgmxF2gTKFWINHgT4GgQ4qjBM2ekKBS?= =?us-ascii?q?UKBESeCRS4+glwBAQIBhHOCXwSBVWsGBzkoIg0MCA4CIAIuB20VBCYvEy0Ck?= =?us-ascii?q?AMHJSuCOwGmKIMGgy+GEI9sgmkFBwMfgzeKT4VPkAGgEpIPhHqBayGBWTMaC?= =?us-ascii?q?BsVgyRQGQ2OKw0JiGGFRkADLwI2AgYBCQEBAwmMEwEB?=
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-AV: E=Sophos;i="5.81,222,1610409600"; d="scan'208";a="33849426"
Received: from aer-iport-nat.cisco.com (HELO aer-core-3.cisco.com) ([173.38.203.22]) by aer-iport-4.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA; 04 Mar 2021 11:28:47 +0000
Received: from [10.60.140.52] (ams-ppsenak-nitro3.cisco.com [10.60.140.52]) by aer-core-3.cisco.com (8.15.2/8.15.2) with ESMTP id 124BSkRq000917; Thu, 4 Mar 2021 11:28:47 GMT
To: Robert Raszuk <robert@raszuk.net>
Cc: Gyan Mishra <hayabusagsm@gmail.com>, Rajesh M <mrajesh@juniper.net>, Shraddha Hegde <shraddha@juniper.net>, DECRAENE Bruno IMT/OLN <bruno.decraene@orange.com>, Tony Li <tony.li@tony.li>, "lsr@ietf.org" <lsr@ietf.org>, William Britto A J <bwilliam=40juniper.net@dmarc.ietf.org>
References: <161401476623.19237.3808413288895066510@ietfa.amsl.com> <CAOj+MMGZppwYtNr4t0rJoy3BKWaBYqHiJ_esM1XNFTNxbm8c5w@mail.gmail.com> <08882555-009B-4068-ABB0-20B0D165D722@tony.li> <2c2605a8-95c6-a477-b1b5-5ae4d4de222a@cisco.com> <52B3A5ED-6ACC-4772-BEF7-085A33A53F31@tony.li> <e5190522-3a8b-2d6e-c2fe-646049689cc4@cisco.com> <1EABA651-2F05-415B-97EF-054507FADEAC@tony.li> <f935dbc4-6220-5f47-65a4-f642823f594f@cisco.com> <CAOj+MMHXd5j8B9a13E90HQVB=SUOkQ=fqhyJEgTf-Y7Tp5eiBQ@mail.gmail.com> <9d66e5af-414a-42b7-6af5-388974785b8f@cisco.com> <CAOj+MMGggyuzqD=ZbDxNG_10ThHykOO7y5rYWphvSk0rs+4CbQ@mail.gmail.com> <bb9555c4-8007-4d0e-2715-39c4069fc61f@cisco.com> <CAOj+MMFP+Tx13v99GaOc8vQaN6UPkSiS7UX1hRFwevKd8Oj13Q@mail.gmail.com> <532100cf-0252-73f3-046c-218de2fc26db@cisco.com> <CAOj+MMEYAc=LN6JfTto7fwcSzeQuyUdeTTC262c5LwBfPFgRRg@mail.gmail.com> <f21c7460-86ff-6169-1bf8-3e95e843e9df@cisco.com> <CAOj+MMHtFS3X7G622ik3q3uy46M=49dY3gQ60X8pCxNyga9QLw@mail.gmail.com>
From: Peter Psenak <ppsenak@cisco.com>
Message-ID: <528e48f4-3fa9-c9bb-b707-3c451aa05eac@cisco.com>
Date: Thu, 4 Mar 2021 12:28:46 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Thunderbird/60.9.1
MIME-Version: 1.0
In-Reply-To: <CAOj+MMHtFS3X7G622ik3q3uy46M=49dY3gQ60X8pCxNyga9QLw@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-Outbound-SMTP-Client: 10.60.140.52, ams-ppsenak-nitro3.cisco.com
X-Outbound-Node: aer-core-3.cisco.com
Archived-At: <https://mailarchive.ietf.org/arch/msg/lsr/hBFMBEnl9wnjssVVoCxanowMwTA>
Subject: Re: [Lsr] New draft on Flex-Algorithm Bandwidth Constraints
X-BeenThere: lsr@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Link State Routing Working Group <lsr.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/lsr>, <mailto:lsr-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/lsr/>
List-Post: <mailto:lsr@ietf.org>
List-Help: <mailto:lsr-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/lsr>, <mailto:lsr-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 04 Mar 2021 11:29:00 -0000

On 04/03/2021 12:25, Robert Raszuk wrote:
> Ok so there are some agreements. Good.
> 
> As to the *minimum* you keep highlighting IMO minimum without the 
> definition of what time window this minimum is related to is meaningless.
> 
> Is it minimum ever, is it min of the year, month, week, day ... etc

https://tools.ietf.org/html/rfc8570#section-4.2

Peter


> 
> Best
> R.
> 
> On Thu, Mar 4, 2021, 12:06 Peter Psenak <ppsenak@cisco.com 
> <mailto:ppsenak@cisco.com>> wrote:
> 
>     On 04/03/2021 11:52, Robert Raszuk wrote:
>      > I guess now you are not listening ;)
>      >
>      > I am saying it is about finding the right normalization timers and
>      > values which can satisfy the need. And those should reside on the
>     senders.
> 
>     that's about what tend to agreed on so far.
> 
>     But that will not address the entire problem. No timers would make it
>     work in environment where the delay fluctuates constantly and
>     significantly. There timers would only make sure the stability is not
>     compromised. But the delay optimized forwarding will not be achieved.
> 
>      >
>      > That is why I was asking what is there today and so far no one gave
>      > precise answer.
> 
>     I did provide a pretty detailed answer about what is there in one
>     implementation.
> 
> 
>      > You said links don't change delay characteristics which
>      > by itself only applies to some link categories. Not satellite not
>     even
>      > 5G.Leave alone VPWS.
>      >
>      > So I guess your conclusion is that this is hard problem and we
>     should
>      > not go there. But if so let's not pretend we are sending even min
>     link
>      > delay as dynamic value and redefine it as static approximated
>     link delay.
> 
>     no, I did not conclude anything. I was providing a reasoning of why min
>     delay was chosen versus another, even more dynamic values.
> 
>     Peter
>      >
>      > Thx
>      > R.
>      >
>      >
>      >
>      >
>      >
>      >
>      > On Thu, Mar 4, 2021, 11:06 Peter Psenak <ppsenak@cisco.com
>     <mailto:ppsenak@cisco.com>
>      > <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>>> wrote:
>      >
>      >     Robert,
>      >
>      >     On 04/03/2021 10:50, Robert Raszuk wrote:
>      >      > Peter,
>      >      >
>      >      > Sorry but the draft does not have a disclaimer section
>     which states:
>      >      >
>      >      > "Extensions defined below are only applicable to networks with
>      >     not even
>      >      > a single emulated circuit in the IGP."
>      >      >
>      >      >  > Obviously, if the min delay fluctuates wildly, one can not
>      >     achieve
>      >      > delay optimized
>      >      >  > forwarding no matter what.
>      >      >
>      >      > If VPWS provider's IP network reconverges once per day or per
>      >     hour I see
>      >      > nothing wrong with flooding new link delays and
>     recomputing the
>      >     topology
>      >      > of the network running on top of such constructs.
>      >
>      >     the question is how much sense would it make to try to optimize
>      >     based on
>      >     delay if it fluctuates every ms and we optimize once per day.
>     But feel
>      >     free to give it a shot if you believe it's a good idea.
>      >
>      >     Peter
>      >
>      >      > My point is that
>      >      > changes are real, pretty unpredictable and in ms or 10s of ms.
>      >     And IMO
>      >      > the proposal on the table is specifically designed to
>     catch and
>      >     address
>      >      > those cases - not just dismiss it as you can not use it in
>     your
>      >     network
>      >      > - sorry mr customer. >
>      >      > Best,
>      >      > R.
>      >      >
>      >      >
>      >      >
>      >      >
>      >      >
>      >      >
>      >      > On Thu, Mar 4, 2021 at 10:41 AM Peter Psenak
>     <ppsenak@cisco.com <mailto:ppsenak@cisco.com>
>      >     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>>
>      >      > <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>
>     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>>>> wrote:
>      >      >
>      >      >     Robert,
>      >      >
>      >      >     On 04/03/2021 10:23, Robert Raszuk wrote:
>      >      >      > Peter,
>      >      >      >
>      >      >      > Completely disagree.
>      >      >
>      >      >     that's what you seem to be doing from the beginning of
>     this
>      >      >     conversation ;)
>      >      >
>      >      >      >
>      >      >      > Real say enterprise networks are being build with
>     emulated
>      >     circuits
>      >      >      > (example VPWS). In one company I was working at it was
>      >     about 80% of
>      >      >      > emulated links all over the world in their WAN. Yes
>     for me
>      >     it was a
>      >      >      > shock as I did not realize how much this emulated links
>      >     took over
>      >      >     the
>      >      >      > world. In most geographies you even can not get any
>     link
>      >     of less
>      >      >     then
>      >      >      > 10Gbps to be real any more. Only emulated option is
>     on the
>      >     table.
>      >      >      >
>      >      >      > Emulated circuits run over someone's IP backbones.
>     You can
>      >      >     understand
>      >      >      > the consequences of this. Not only link delay changes a
>      >     lot but
>      >      >     you run
>      >      >      > into very interesting set of issues.
>      >      >
>      >      >     look at it from the opposite direction. The provider
>     of the
>      >     VPWS is the
>      >      >     one who can use this technology to guarantee the delay
>     bound
>      >     of the
>      >      >     WPWs
>      >      >     service. And if it does, the user of the WPWs would not
>      >     experience the
>      >      >     wild variation in the min delay.
>      >      >
>      >      >     So you have to apply right set of tools at right location.
>      >      >     Obviously, if
>      >      >     the min delay fluctuates wildly, one can not achieve delay
>      >     optimized
>      >      >     forwarding no matter what. That does not mean that the
>      >     network will get
>      >      >     unstable.
>      >      >
>      >      >     Peter
>      >      >
>      >      >
>      >      >      >
>      >      >      > Maybe you think of the backbones of the 90s or
>     2000s where
>      >     dark
>      >      >     fiber or
>      >      >      > SDH or SONET were in use for interconnects.
>      >      >      >
>      >      >      > Well no more. IETF came with such brilliant ideas to
>      >     emulate L2
>      >      >     over L3
>      >      >      > and here we go.
>      >      >      >
>      >      >      > Reality is not what we wish it to be.
>      >      >      >
>      >      >      > Cheers,
>      >      >      > R.
>      >      >      >
>      >      >      >
>      >      >      > On Thu, Mar 4, 2021 at 10:07 AM Peter Psenak
>      >     <ppsenak@cisco.com <mailto:ppsenak@cisco.com>
>     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>>
>      >      >     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>
>     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>>>
>      >      >      > <mailto:ppsenak@cisco.com
>     <mailto:ppsenak@cisco.com> <mailto:ppsenak@cisco.com
>     <mailto:ppsenak@cisco.com>>
>      >     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>
>     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>>>>> wrote:
>      >      >      >
>      >      >      >     Robert,
>      >      >      >
>      >      >      >     On 03/03/2021 20:57, Robert Raszuk wrote:
>      >      >      >      > Peter,
>      >      >      >      >
>      >      >      >      >  >  that differ by few microsecond
>      >      >      >      >
>      >      >      >      > Really you normalize only single digit
>     microseconds ???
>      >      >      >      >
>      >      >      >      > What if link delay changes in
>     milliseconds scale ?
>      >     Do you
>      >      >     want to
>      >      >      >      > compute new topology every few milliseconds ?
>      >      >      >
>      >      >      >     let me repeat again.
>      >      >      >
>      >      >      >     Min delay is not something that changes every few
>      >     milliseconds
>      >      >      >     significantly. It's a semi static variable that
>      >     reflects the
>      >      >      >     property of
>      >      >      >     the underlying physical path. It only changes
>     when the
>      >      >     physical path
>      >      >      >     properties changes - e.g. the optical path
>     reroutes,
>      >     etc. We
>      >      >      >     deliberately picked Min delay for flex-algo
>     purposes
>      >     for this
>      >      >     semi
>      >      >      >     static property.
>      >      >      >
>      >      >      >     The small, non significant changes can be
>     filtered by the
>      >      >     normalization.
>      >      >      >
>      >      >      >     If the min delay changes significantly every few
>      >     milliseconds
>      >      >     there's
>      >      >      >     something wrong with the link itself - we have
>      >     standard dampening
>      >      >      >     mechanisms in LS protocols to deal with
>     unstable links
>      >     that
>      >      >     would kick
>      >      >      >     in. Similar to what we do if the link flaps
>     every few
>      >      >     milliseconds.
>      >      >      >
>      >      >      >      >
>      >      >      >      > Out of curiosity as this is not a secret -  What
>      >     are your
>      >      >     default
>      >      >      >     min
>      >      >      >      > delay normalization timers (if user does not
>     overwrite
>      >      >     with their
>      >      >      >     own).
>      >      >      >
>      >      >      >     there is no timer needed for the normalization
>     itself.
>      >      >      >
>      >      >      >     You are likely referring TWAMP computation interval
>      >     which is
>      >      >     30 sec,
>      >      >      >     with probes being sent every 3 seconds in our
>      >     implementation by
>      >      >      >     default,
>      >      >      >     if I'm not mistaken.
>      >      >      >
>      >      >      >     Normalization is applied to the value that come
>     from
>      >     the above.
>      >      >      >
>      >      >      >     thanks,
>      >      >      >     Peter
>      >      >      >
>      >      >      >
>      >      >      >
>      >      >      >      > Likewise how Junos or Arista normalizes it
>     today ?
>      >      >      >      >
>      >      >      >      > Thx,
>      >      >      >      > R.
>      >      >      >      >
>      >      >      >      > On Wed, Mar 3, 2021 at 7:41 PM Peter Psenak
>      >      >     <ppsenak@cisco.com <mailto:ppsenak@cisco.com>
>     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>>
>      >     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>
>     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>>>
>      >      >      >     <mailto:ppsenak@cisco.com
>     <mailto:ppsenak@cisco.com> <mailto:ppsenak@cisco.com
>     <mailto:ppsenak@cisco.com>>
>      >     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>
>     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>>>>
>      >      >      >      > <mailto:ppsenak@cisco.com
>     <mailto:ppsenak@cisco.com>
>      >     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>>
>     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>
>      >     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>>>
>      >      >     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>
>     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>>
>      >     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>
>     <mailto:ppsenak@cisco.com <mailto:ppsenak@cisco.com>>>>>> wrote:
>      >      >      >      >
>      >      >      >      >     Tony,
>      >      >      >      >
>      >      >      >      >     On 03/03/2021 19:14, Tony Li wrote:
>      >      >      >      >      >
>      >      >      >      >      > Peter,
>      >      >      >      >      >
>      >      >      >      >      >>> There are several link types in use
>     that
>      >     exhibit
>      >      >     variable
>      >      >      >      >     delay: satellite links (e.g., Starlink),
>     microwave
>      >      >     links, and
>      >      >      >      >     ancient link layers that deliver reliability
>      >     through
>      >      >      >     retransmission.
>      >      >      >      >      >>> Any of these (and probably a lot
>     more) can
>      >     create a
>      >      >      >     noticeable
>      >      >      >      >     and measurable difference in TWAMP. That
>     would be
>      >      >     reflected
>      >      >      >     in an FA
>      >      >      >      >     metric change. If you imagine a
>     situation with
>      >      >     multiiple parallel
>      >      >      >      >     paths with nearly identical delays, you
>     can easily
>      >      >     imagine an
>      >      >      >      >     oscillatory scenario.   IMHO, this is an
>      >     outstanding
>      >      >     concern
>      >      >      >     with FA.
>      >      >      >      >      >> yes, and that is what I referred to
>     as "delay
>      >      >     normalization",
>      >      >      >      >     which can avoid that oscillation.
>      >      >      >      >      >
>      >      >      >      >      >
>      >      >      >      >      > It can also negate the benefits of the
>      >     feature. One
>      >      >     might well
>      >      >      >      >     imagine that Starlink would want to follow a
>      >     min-delay
>      >      >     path for
>      >      >      >      >     optimality.  If the delay variations are
>      >     “normalized”
>      >      >     out of
>      >      >      >      >     existence, then the benefits are lost. 
>     The whole
>      >      >     point is to
>      >      >      >     track
>      >      >      >      >     the dynamics.
>      >      >      >      >
>      >      >      >      >     for all practical purposes that we use
>     it for,
>      >     the two
>      >      >     values
>      >      >      >     of min
>      >      >      >      >     delay that differ by few microsecond can be
>      >     treated as
>      >      >     same
>      >      >      >     without any
>      >      >      >      >     loss of functionality. So it's about the
>     required
>      >      >     normalization
>      >      >      >      >     interval
>      >      >      >      >     - something that can be controlled by
>     the user.
>      >      >      >      >
>      >      >      >      >     thanks,
>      >      >      >      >     Peter
>      >      >      >      >
>      >      >      >      >
>      >      >      >      >
>      >      >      >      >      >
>      >      >      >      >      >
>      >      >      >      >      >>> Please note that I’m NOT
>     recommending that we
>      >      >     back away.
>      >      >      >      >     Rather, we should seek to solve the
>      >     long-standing issue of
>      >      >      >      >     oscillatory routing.
>      >      >      >      >      >>
>      >      >      >      >      >> not that I disagree. History tells
>     us that
>      >     the generic
>      >      >      >     case of
>      >      >      >      >     oscillation which is caused by the traffic
>      >     itself is a
>      >      >     hard
>      >      >      >     problem
>      >      >      >      >     to solve.
>      >      >      >      >      >
>      >      >      >      >      >
>      >      >      >      >      > Any oscillation is difficult to
>     solve.  Positive
>      >      >     feedback
>      >      >      >      >     certainly can exacerbate the problem. But
>      >     solving hard
>      >      >      >     problems is
>      >      >      >      >     why we are here.
>      >      >      >      >      >
>      >      >      >      >      > Yours in control theory,
>      >      >      >      >      > Tony
>      >      >      >      >      >
>      >      >      >      >      >
>      >      >      >      >      >
>      >      >      >      >
>      >      >      >
>      >      >
>      >
>