Re: [Tsv-art] [Lsr] Tsvart early review of draft-ietf-lsr-isis-fast-flooding-06

[[e-mail]]

Hi Mirja,

Thanks for your follow up on this and your time
Please see inline [Bruno3]
 
> -----Original Message-----
> From: Mirja Kuehlewind (IETF) <[e-mail]> 
> Sent: Thursday, February 29, 2024 3:30 PM
> To: DECRAENE Bruno INNOV/NET <[e-mail]>
> Cc: Les Ginsberg (ginsberg) <ginsberg=[e-mail]>; [e-mail]; [e-mail]; [e-mail]; [e-mail]
> Subject: Re: [Lsr] Tsvart early review of draft-ietf-lsr-isis-fast-flooding-06
> 
> Hi Bruno,
> 
> Sorry for my late reply.
> 
> Please see some more comments below.
> 
> > On 9. Feb 2024, at 21:50, [e-mail] wrote:
> > 
> > Hi Mirja,
> >  
> > Thanks for your replies. Please see inline [Bruno2]
> >  
> > On a side note, we have presented some tests results at IETF 111. If you want to have a look at them, please find below the slides. If you have some comments on the tests results, I would be obviously interested in your comments. Either on the list or of the list.
> > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdata
> > tracker.ietf.org%2Fmeeting%2F111%2Fmaterials%2Fslides-111-lsr-22-flow-
> > congestion-control-00.pdf&data=05%7C02%7Cbruno.decraene%40orange.com%7
> > C2a11513881214cd239fd08dc3932fc06%7C90c7a20af34b40bfbc48b9253b6f5d20%7
> > C0%7C0%7C638448138656092668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMD
> > AiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=A2
> > cRtfmaAgWVdE3S0KeNnlfyQfEgNsxIxHSNowwOQe0%3D&reserved=0
> >  
> >  
> >  
> > > From: Lsr <[e-mail] <mailto:[e-mail]>> On 
> > > Behalf Of Mirja Kuehlewind (IETF)
> > > Sent: Thursday, February 8, 2024 2:06 PM
> > > 
> > > Hi Bruno,
> > > 
> > > Thanks for your replies.
> > > 
> > > On the high-level I think that some or most of the explanation you provide me below about parameter values, should actually go into the draft. I understand that there is not a one fits all but that's why min/max values are often more important than recommended values. Yes, these might also not hold forever but maybe that just mean there is a point in future where it makes sense to update this RFC. Also you say it depends on the performance/capability of the router, however, I think you can say something like: with an average router today with these performance parameters, these are our tested values that showed good performance and also this and that value can e.g. scale with e.g. more CPU power. Something like this.
> > > 
> > > See further below.
> >  
> > [Bruno2] OK. See further below.
> >  
> > > > On 5. Feb 2024, at 19:17, [e-mail] <mailto:[e-mail]> wrote:
> > > > 
> > > > [+Les, Guillaume as we go quite deep in the discussion]
> > > > 
> > > > Hi Mirja,
> > > > 
> > > > Thank you for your review and comments. Very useful.
> > > > 
> > > > Please see inline [Bruno]
> > > > 
> > > >> From: Mirja Kühlewind via Datatracker <[e-mail] 
> > > >> <mailto:[e-mail]>>
> > > >> Sent: Friday, February 2, 2024 3:57 PM
> > > >> 
> > > >> Reviewer: Mirja Kühlewind
> > > >> Review result: Not Ready
> > > >> 
> > > >> First of all I have a clarification question: The use the of flags TLV with the O flag is not clear to me. Is that also meant as a configuration parameter or is that supposed to be a subTLV that has to be sent together with the PSNP? If it is a configuration, doesn't the receiver need to confirm that the configuration is used and how does that work in the LAN scenario where multiple configurations are used? If it has to be sent together with the PSNP, this needs to be clarified and it seem a bit strange to me that it is part of the same TLV. Or maybe I'm missing something completely about the flag?
> > > > 
> > > > [Bruno] The O-flag is advertised by the receiver in the Flags sub-TLV, which may be sent either in PSNP or IIH.
> > > > That's not a configuration but a capability of the receiver which is signaled to the sender.
> > > > That's only applicable to the point-to-point scenario, not the LAN scenario. ( as on a LAN there is no explicit acknowledgment of the receipt of LSPs between a given LSP transmitter and a given LSP receiver).
> > > > 
> > > >> it seem a bit strange to me that it is part of the same TLV
> > > > 
> > > > [Bruno]
> > > > All those sub-TLVs, at least the one currently defined, carries
> > > > (relatively) static parameters and are not required to be sent in all IIH or PSNP. The way IS-IS acknowledges the reception of LSP is not changed They are all grouped in a single TLV called " Flooding Parameters TLV" for grouping purpose and also because IS-IS has a limited TLV space.
> > > > If the above does not clarify, could you please elaborate on what you feel "strange" about?
> > > 
> > > Okay, it a capability. Does that mean if the capability is announced the routing that has sent the announcement will consider order in all PSNP? I think this could be stated more clearly.
> >  
> > [Bruno2] Correct, that's a capability but also a commitment to do it.
> > Would the following rephrase help?
> >  
> > OLD:
> > When the O-flag (Ordered acknowledgement) is set, the LSPs will be 
> > acknowledged in the order they are received: a PSNP acknowledging N 
> > LSPs is acknowledging the N oldest LSPs received. The order inside the 
> > PSNP is meaningless. If the sender keeps track of the order of LSPs 
> > sent, this indication allows a fast detection of the loss of an LSP. 
> > This MUST NOT be used to alter the retransmission timer for any LSP. 
> > This MAY be used to trigger a congestion signal.</t>
> >  
> > NEW:
> > When setting the O-flag, the LSP receiver MUST acknowledge LSPs in the same order than it has received the LSPs. Therefore, a PSNP acknowledging N LSPs is acknowledging the N oldest received LSPs. Note that the order of LSP-IDs inside the PSNP is meaningless.
> > The LSP sender MAY use this information to faster detect the loss of an LSP and trigger a congestion signal. it MUST NOT use this information to alter the retransmission timer for any LSP.
> >  
> I guess the point that is not clear to me is, how does the sender of the LSP know that the receiver of the LCP understands the O-flag and has processed it accordingly? Maybe I missed something?

[Bruno3] The O-flag is advertised by the receiver of the LSPs. Hence if the sender see the O-flag from the receiver, it knows that the receiver acknowledges the LSP in order.
Then the sender MAY use that information to detect that the receiver has not received an LSP (the one not acknowledged) and assume that this is due to some congestion on the way.

Note that the proposed NEW is not reflected in -07. I was waiting for your feedback on whether the NEW helped. Apparently, not good enough. If you have suggestion this may help.


> 
> >  
> > > > 
> > > > 
> > > >> Then, generally thank you for considering overload and congestion carefully.
> > > >> Please see my many comments below, however, I think one important part is to ensure that the network/link doesn't get normally overloaded with the parameter selected. You give some recommendation about parameters to use but not for all and, more importantly, it would be good to define boundaries for safe use.
> > > >> What's a "safe" min or max value? I know this question is often not easy to answer, however, if you as the expert don't give the right recommendations, how should a regular implementer make the right choice?
> > > > 
> > > > [Bruno]
> > > > Very fair points. And thank you for acknowledging that this question is not easy to answer...
> > > > TL;DR: sorry, I don't know.
> > > > 
> > > > Two general statements first:
> > > > - IS-IS is running in the control plane of two adjacent routers, typically in backbone of network operators. There is a single point to point link over fiber with typically latest interfaces speed (e.g. > 100G today). I would not assume that IS-IS would overload, or even significantly load this interface. From a jitter standpoint packet priority/CoS could be discussed but I would assume that I'm assuming that this is a different discussion.
> > > > - currently IS-IS has no flow control nor congestion control. Given this, current values are very conservative (e.g., one packet every 33ms). At the same time that's a very important signaling for the network: we would prefer not dropping LSP but on the other hand delaying LSP for seconds is not helping. For historically and good reasons IS-IS implementers are very conservative. As of today, I would not assume that they would be too aggressive.
> > > > - one problem of stating values in RFC is that those values may not age well. That's typically the case with IS-IS with some parameters values which are still 25 years old while CPU and networks have evolved significantly since then. So I'm a bit reluctant to write static values in stone again.
> > > > 
> > > > Coming back to min and max value:
> > > > - I'm not an implementor, but I do care about my networks. If I were an implementor, I would play safe and advertise values which are safe for my implementation as receiver. I would rather use those values as a protection from a too aggressive sender, than as a permission to overload (DOS) me. Both sender and receiver are within the same administrative domain (for certain). In case of issue, the network operator will debug both the sender and receiver and blame the one which did not behave.
> > > > - There are a wide range router size/price/capability/generation. Plus I would expect this value to gradually improve as the implementation improve thanks to the capabilities introduced by this document.
> > > > - I'm definitely not a transport/TCP expert. Does TCP define such min and max values?
> > > 
> > > Yes and no. TCP has a default ACK rate of 2. There is no recommendation on e.g. the receive window, however, for TCP the purpose it to fully utilise the link and the receive window can be adjusted overtime depending on the current congestion window (and local load). Also there is no fixed pacing rate; pacing is dynamically calculated based on RTT and the congestion window. So in short I don't think this is comparable as many of the parameter are not a fixed configuration and that's because is not only about avoiding overload but also maximising throughput.
> >  
> > [Bruno2] OK, IS-IS history is a bit different than the TCP one As 
> > indicated in §3, IS-IS implementations had "historical behaviors" with 
> > static parameters configured on the LSP sender, irrespectively of the capability of the receiver (although they represent limitations of the receiver) This document allows two things:
> > -a- the signaling of those current parameters by the receiver. That's useful now and easy to do.
> > -b- new flow control and congestion control behavior. Using the new Receive Window sub-TLV and requiring improvements on the received (§5). That's more medium term in multi vendor networks as both the sender and the receiver needs to be upgraded (although, this may depend on what implementations already do) Just like with TCP I believe, different congestion control may be specified for the sender.
> >  
> > In retrospect, may be having two separate documents would have been easier.
> >  
> > Coming back to "safe" values. I'm a bit reluctant to indicate values 
> > which may quickly be outdated, especially given that TCP does not do 
> > it either, but I can live with adding typical acceptable range in 
> > section  6.2.4. Determining values to be advertised in the Flooding 
> > Parameters TLV Something like, to be added after the second paragraph
> 
> As you say there are two parts, one is setting stair parameters. The other is dynamically adjusting congestion control. Note that TCP always has congestion control. However, for you case a) you still need to give some recommendation to implementor otherwise they will get it wrong. It better to be too conservative than to aggressive. Also if you described what the assumption are for these recommendation e.g. in relation to CPU capabilities, implementors can adjust accordingly when these assumption change in future. I think there is still a lack of both in the comment: 1) making default recommendations or at least discussion approbate value ranges and 2) clearly spell out the assumptions made rather then just recommending random number (where something is recommended).
> 
> >  
> > NEW:  Static values are dependent on the CPU generation, class of router and network scaling, typically the number of adjacent neighbors. As examples at the time of publication, LSP Burst Size could be in the range 5 to 20, LSP Transmission Interval in the range of 1ms to 33 ms, LPP in the range of 5 to 90 with a proposed 15. PartialSNPInterval in the range 50ms to 500ms with a proposed 200ms. Receive Window in the range of 30 to 200 with a proposed 60. In general, the larger the better the performance on links with high RTT.
> 
> I think it would be good to add these recommendation, however, I don't think I saw this in the new draft version? However, as said above and inline with your concern about outdating, I would recommend to say even more why these? Are there any hard limits (MUST be larger/smaller than)? Which value depends on which things you name in the first sentence. E.g. is there anything that should scale with the number of neighbours? 

[Bruno3] You are correct that the proposed NEW is not reflected in -07. I was waiting for your feedback on the text before applying the change.
Thanks for your suggestions. Based on these I would propose
NEW:
Static values are dependent on the CPU generation, class of router and network scaling, typically the number of adjacent neighbors. Examples at the time of publication are provided below. LSP Burst Size could be in the range 5 to 20. From a router perspective, this value typically depends on the buffer size on the I/O path from the packet forwarding engine to the control plane which is very platform dependent. It also depends upon how many IS-IS neighbors share this I/O path as typically all neighbors will send the same LSPs at the same time. It may also depends on other incoming control plane traffic sharing that I/O path, how much bursty they are, and how much incoming IS-IS packets are prioritized over those other incoming control plane traffic.  As indicated in section 3, the historical behavior from [ISO10589] allow a value of 10 hence 10 seems conservative. From a network operation perspective, it would be beneficial for the burst size to be equal or higher than the number of LSPs which may be originated by a single failure. For a node failure, this is equal to the number of IS-IS neighbors of the failed link. 
LSP Transmission Interval could be in the range of 1 ms to 33 ms. As indicated in section 3, the historical behavior from [ISO10589] is 33ms hence is conservative. The LSP Transmission Interval is an advertisement of the receiver's sustainable LSP reception rate taking into account all aspects and in particular the control plane CPU and the I/O bandwidth. It's expected to improve (hence decrease) as hardware and software naturally improve over time. It should be chosen relatively conservatively as this rate may be used by the sender in all conditions including worst conditions. It's also not a bottleneck as the flow control algorithm may use a higher rate in good conditions, in particular when the receiver acknowledge quickly and the receive window is large enough compared to the RTT.
LPP could be in the range of 5 to 90 with a proposed 15. A smaller value provides faster feedback at the cost of the small overhead of more PSNP messages (yet same global content). The value is not susceptible to be "unsafe". 
PartialSNPInterval could be in the range 50ms to 500ms with a proposed 200ms. One may distinguish the value used locally from the value signaled to the sender. The value used locally benefits from being small but is not expected to be the main parameter to improve performance. It depends on how fast the IS-IS flooding process may be scheduled by the CPU. It's  safe as if the receiver CPU is busy, it will naturally delay its acknowledgments which provides a negative feedback loop. The value advertised to the sender should be conservative (high enough) as this value could be used by the sender to send some LSPs rather than keep waiting for acknowledgments. But sending a value smaller than the [ISO10589] default may be beneficial to faster trigger re-transmission of lost LSPs rather than waiting.  
Receive Window in the range of 30 to 200 with a proposed 60. In general, the larger the better the performance on links with high RTT. The higher the number and the higher the IS-IS neighbor, the higher the use of control plane memory so it's mostly dependent on the amount of memory which may be dedicated to IS-IS flooding and the number of IS-IS neighbors. From a memory usage perspective, a priori, one could use the same value than the TCP receive window, but the value advertised should not be higher than the buffer of the "socket" used (which is typically different from the TCP socket).
 

I'll push the text to our repo, but I'll wait for more feedback from you, the WG of co-authors, so the text may further be updated.

Regards,
--Bruno

> 
> >  
> > FYI, the IS-IS spec use this type of phrasing "A reasonable value is 10 s"
> >  
> > > > 
> > > > We propose some guidance for the LPP (which is somewhat IS-IS
> > > > specific) in
> > > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > data%2F&data=05%7C02%7Cbruno.decraene%40orange.com%7C2a11513881214
> > > > cd239fd08dc3932fc06%7C90c7a20af34b40bfbc48b9253b6f5d20%7C0%7C0%7C6
> > > > 38448138656100610%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ
> > > > IjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=0FrK
> > > > Y7NWP1e7mPNQZ5bJ4KtpkoypQLCKqLd9IDmXAW0%3D&reserved=0 
> > > > <https://data/> tracker.ietf.org 
> > > > <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2F
> > > > tracker.ietf.org%2F&data=05%7C02%7Cbruno.decraene%40orange.com%7C2
> > > > a11513881214cd239fd08dc3932fc06%7C90c7a20af34b40bfbc48b9253b6f5d20
> > > > %7C0%7C0%7C638448138656106245%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4w
> > > > LjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7
> > > > C&sdata=B%2B%2BdizjYduqrlLmG3wDyoPhC7xhGsBqcyjy%2BUk8AuIw%3D&reser
> > > > ved=0>%2Fdoc%2Fhtml%2Fdraft-ietf-lsr-isis-fast-flooding-06%2
> > > > 3section-5.1&data=05%7C02%7Cbruno.decraene%40orange.com 
> > > > <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2F
> > > > 40orange.com%2F&data=05%7C02%7Cbruno.decraene%40orange.com%7C2a115
> > > > 13881214cd239fd08dc3932fc06%7C90c7a20af34b40bfbc48b9253b6f5d20%7C0
> > > > %7C0%7C638448138656110364%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
> > > > MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sd
> > > > ata=1sllQhmRukLjx4UFbUFfAVqgB1nF8F7zvo7jrUj3BY8%3D&reserved=0>%7C7
> > > > 96ca607c751
> > > > 4eeb037d08dc28a6e12e%7C90c7a20af34b40bfbc48b9253b6f5d20%7C0%7C0%7C
> > > > 6384 
> > > > 29945017174074%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo
> > > > iV2l
> > > > uMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=FLHA6NCWwpX
> > > > 9DN2
> > > > RK7K2oicXt5cGNEogmveul5oFYJQ%3D&reserved=0
> > > > But for other parameters, I'm not sure that indicating values would be useful.
> > > > 
> > > >> Please see further comments below.
> > > >> 
> > > >> Section 4.7:
> > > >> "NOTE: The focus of work used to develop the example algorithms discussed later in this document focused on operation over point-to-point interfaces. A full discussion of how best to do faster flooding on a LAN interface is therefore out of scope for this document."
> > > >> 
> > > >> Actually this is quite important and also not clear to me. You do discuss how to interpret parameters in a LAN scenario but then you say you only give proper guidance how to adjust the sending rate for non-LAN. But what's the right thing to do in LAN then? Why is LAN out of scope? If you don't give guidance, I think you have to also say that this mechanism that enables using higher values in this document MUST NOT be used on LAN setups.
> > > > 
> > > > [Bruno] In point-to-point there is one sender for one receiver. In LAN, there are N receivers for 1 sender, and possibly N senders for all/each receiver. The guidance is whether the multiplicative factor is to be handled by the sender or the receiver. Document says that the value is used by the sender as-is, so it's up to the receiver to take into account the number of speaker on the LAN. This guidance seems required for correct semantic.
> > > > 
> > > > Then the TLV may carries different sub-TLVs. Some may be applicable to LAN (e.g., Burst Size).
> > > > Some are less applicable to LAN because the way IS-IS acknowledge LSP is different in LAN and less dynamic. The LAN case is both less frequent those days (if not rare in backbones) and more difficult to handle as IS-IS acknowledges LSP in a slow and less explicit way hence we have a loose feedback loop to use. Eventually, someone could define new sub-TLV or procedure to improve the LAN case therefore I don't think that we should define TLV as not applicable to LAN.
> > > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > data%2F&data=05%7C02%7Cbruno.decraene%40orange.com%7C2a11513881214
> > > > cd239fd08dc3932fc06%7C90c7a20af34b40bfbc48b9253b6f5d20%7C0%7C0%7C6
> > > > 38448138656114042%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ
> > > > IjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=ayAr
> > > > O5K4F%2BI7kOFdp2hng0ea4MH6Z3Kzp2luKLVeCOk%3D&reserved=0 
> > > > <https://data/> tracker.ietf.org 
> > > > <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2F
> > > > tracker.ietf.org%2F&data=05%7C02%7Cbruno.decraene%40orange.com%7C2
> > > > a11513881214cd239fd08dc3932fc06%7C90c7a20af34b40bfbc48b9253b6f5d20
> > > > %7C0%7C0%7C638448138656117657%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4w
> > > > LjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7
> > > > C&sdata=xW8MDrN3g%2F0XahrQ4oZ2fFvO4keH7%2Bufyp%2BbAcwI3p8%3D&reser
> > > > ved=0>%2Fdoc%2Fhtml%2Fdraft-ietf-lsr-isis-fast-flooding-06%2
> > > > 3section-6.2.1.2&data=05%7C02%7Cbruno.decraene%40orange.com 
> > > > <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2F
> > > > 40orange.com%2F&data=05%7C02%7Cbruno.decraene%40orange.com%7C2a115
> > > > 13881214cd239fd08dc3932fc06%7C90c7a20af34b40bfbc48b9253b6f5d20%7C0
> > > > %7C0%7C638448138656121272%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
> > > > MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sd
> > > > ata=gHYI2JiMvLFynNKgPuXniio2LXB1hIvno4KHP7%2BICxM%3D&reserved=0>%7
> > > > C796ca607 
> > > > c7514eeb037d08dc28a6e12e%7C90c7a20af34b40bfbc48b9253b6f5d20%7C0%7C
> > > > 0%7C 
> > > > 638429945017182027%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJ
> > > > QIjo 
> > > > iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=vAQkKFN
> > > > MlgZ
> > > > zdjvjOmI4Z5fFOMSKCes3kIDPQFaNOo8%3D&reserved=0 does partially 
> > > > discuss and cover the LAN case. Possibly the operation would be 
> > > > further improved, but I think that it's too late to add 
> > > > specification for the LAN . This may be covered in a subsequent 
> > > > doc. (but really LAN is not a priority those days)
> > > 
> > > As I said above, I think either you need to provide appropriate and equavent guidance for LAN or you make to restrict this extension to non-LAN scenarios and say that explicitly in the (like "MUST NOT be used in LAN scenarios").
> >  
> > [Bruno2]
> > The sub-TLVs may be used in LAN and the LAN case is defined in §4.7
>
____________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.