Re: [Lsr] Flow Control Discussion for IS-IS Flooding Speed

["Les Ginsberg (ginsberg)" <[e-mail]>]

Bruno -

Returning to this old thread...

The following is a generic description of how receive path for IS-IS PDUs functions today - based on examination of implementations on a variety of platforms from my employer. It is, for obvious reasons,  generic and intentionally omits any discussion of implementation details.
But it is hopefully detailed enough to illustrate some of the challenges in regards to providing real-time  accurate feedback to the control plane that could be used by a receiver based flow control mechanism.

Stage 1: Input Policing

As a first step, a policer operates at ingress to rate limit the number of packets which will be punted for local processing. This typically operates in an interface independent manner - limiting the total number of packets per second across all interfaces.
The policer may rate limit different classes of packets or simply limit all types of packets. A given platform may apply a limit specific to IS-IS PDUs or apply a limit to a class of packets (e.g., OSPF+BGP+IS-IS combined).

Stage 2: Punt Queue Shaping

Received packets are then placed in a queue for eventual transfer to the control plane. The number of queues varies. In some cases a single queue for all packet types is used. In other cases packets are placed on different queues based on packet classes.
Each queue is typically bounded to a maximum number of packets. As the queue usage approaches a limit, shaping policies are applied to prioritize certain packet types over others.
An upper limit specific to IS-IS packets may be employed - or a limit may be applied to a larger class of packet types of which IS-IS is only one of the packet types in the class.

Stage 3: Transfer to control Plane

Packets are then transferred to the control plane. Control plane input queues may map 1-1 with the data plane queues or map many to 1.
If the incoming packets are encapsulated (for example GRE) they may be transferred to a media specific control plane queue to process the encapsulation header. In some cases encapsulation may be processed in the data plane prior to transfer to the control plane.

Stage 4: Transfer to IS-IS

Packets are then transferred to a queue which is read directly by IS-IS. In the event there are multiple IS-IS instances, implementations may choose to have a shared queue which drives the execution of all instances or have instance specific queues filtered based on incoming interface.

A single queue is typically used for all interfaces and all IS-IS packet types. Subsequent processing may requeue packets based on packet type e.g., separating processing of hellos from processing of LSPs/SNPs.

*********

A receiver based flow control mechanism which attempts to make dynamic adjustments needs to obtain real-time feedback from one or more of the above stages. Monitoring the state of the input queue to IS-IS is easy to do, but would not account for drops at previous stages.
Obtaining feedback from earlier stages requires real-time updates from data plane to IS-IS in the control plane. This is much more challenging. Idiosyncrasies of platforms will have a significant impact on how to meaningfully interpret and use the data. How to integrate data from the various stages - especially when the numbers are not specific to IS-IS packets - is not intuitive.

https://tools.ietf.org/html/draft-decraene-lsr-isis-flooding-speed-03 provides no guidance on how information from the dataplane could be used.
As an alternative, it suggests that static parameters derived from offline tests could be advertised. But static parameters would necessarily have to be conservative as they would need to account for the worst case scenarios.

draft-ginsberg-lsr-isis-flooding-scale proposes dynamic flow control based on the state of the transmitter. In this model, there is no dependency on platform implementation. The number of unacknowledged LSPs sent on an interface is used as input to the flow control algorithm. This accounts for all reasons why a receiver may be slow to acknowledge without requiring knowledge of which stage(s) described above are affecting the receiver's ability to provide timely acknowledgements.

   Les


From: [e-mail] <[e-mail]>
Sent: Wednesday, February 26, 2020 11:03 AM
To: Les Ginsberg (ginsberg) <[e-mail]>
Cc: [e-mail]
Subject: RE: Flow Control Discussion for IS-IS Flooding Speed

Les,

Please see inline[Bruno]

From: Lsr [mailto:[e-mail]] On Behalf Of Les Ginsberg (ginsberg)
Sent: Wednesday, February 19, 2020 3:32 AM
To: [e-mail]<mailto:[e-mail]>
Subject: Re: [Lsr] Flow Control Discussion for IS-IS Flooding Speed

Base protocol operation of the Update process tracks the flooding of
LSPs/interface and guarantees timer-based retransmission on P2P interfaces
until an acknowledgment is received.

Using this base protocol mechanism in combination with exponential backoff of the
retransmission timer provides flow control in the event of temporary overload
of the receiver.

This mechanism works without protocol extensions, is dynamic, operates
independent of the reason for delayed acknowledgment (dropped packets, CPU
overload), and does not require additional signaling during the overloaded
period.

This is consistent with the recommendations in RFC 4222 (OSPF).

Receiver-based flow control (as proposed in https://datatracker.ietf.org/doc/draft-decraene-lsr-isis-flooding-speed/ )
requires protocol extensions and introduces additional signaling during
periods of high load. The asserted reason for this is to optimize throughput -
but there is no evidence that it will achieve this goal.

Mention has been made to TCP-like flow control mechanisms as a model - which
are indeed receiver based. However, there are significant differences between
TCP sessions and IGP flooding.

TCP consists of a single session between two endpoints. Resources
(primarily buffer space) for this session are typically allocated in the
control plane and current usage is easily measurable..

IGP flooding is point-to-multi-point, resources to support IGP flooding
involve both control plane queues and dataplane queues, both of which are
typically not per interface - nor even dedicated to a particular protocol
instance. What input is required to optimize receiver-based flow control is not fully specified.

https://datatracker.ietf.org/doc/draft-decraene-lsr-isis-flooding-speed/ suggests (Section 5) that the values
to be advertised:

"use a formula based on an off line tests of
   the overall LSPDU processing speed for a particular set of hardware
   and the number of interfaces configured for IS-IS"

implying that the advertised value is intentionally not dynamic. As such,
it could just as easily be configured on the transmit side and not require
additional signaling. As a static value, it would necessarily be somewhat
conservative as it has to account for the worst case under the current
configuration - which means it needs to consider concurrent use of the CPU
and dataplane by all protocols/features which are enabled on a router - not all of whose
use is likely to be synchronized with peak IS-IS flooding load.
[Bruno] _Assuming_ that the parameters are static, those parameters

     *   are the same as the one implemented (configured) on multiple implementations, including the one from your employer. Now do you believe that those parameters can be determined?

        *   If yes, how do you do _today_ on your implementation? (this seems to contradict your statement that you have no way to figure out how to find the right value)
        *   If no, why did you implement those parameters, and ask network operator to configure them?
        *   There is also the option to reply: I don't know but don't care as I leave the issue to the network operator.

     *   can still provide some form of dynamicity, by using the PSNP as dynamic acknowledgement.
     *   are really dependent on the receiver, not the sender.

        *   the sender will never overload itself.
        *   The receiver has more information,  knowing its processing power (low end, high end, 80s, 20s (currently we are stuck with 20 years old value assuming the worst possible receiver (and worst there were, including with packet processing partly done in the control plane processor)), its expected IS-IS load (#neighbors), its preference for bursty LSP reception (high delay between IS-IS CPU allocation cycles, memory not an issue up to x kilo-octet...), its expected control plane load if IS-IS traffic has not higher priority over other control plane traffic...), it's expected level of QoS prioritization [1]

           *   [1] looks for "Extended SPD Headroom". E.g. "Since IGP and link stability are more tenuous and more crucial than BGP stability, such packets are now given the highest priority and are given extended SPD headroom with a default of 10 packets. This means that these packets are not dropped if the size of the input hold queue is lower than 185 (input queue default size + spd headroom size + spd extended headroom)."

              *   And this is for distributed architecture, 15 years ago. So what about using the above number (in the router configuration), applies Tony's proposal (*oversubscription/number of IS neighbhors), and advertise this value to your LSP sender?



[1] https://www.cisco.com/c/en/us/support/docs/routers/12000-series-routers/29920-spd.html


  *
--Bruno


Unless a good case can be made as to why transmit-based flow control is not a good
fit and why receiver-based flow control is demonstrably better, it seems
unnecessary to extend the protocol.

    Les


From: Lsr <[e-mail]<mailto:[e-mail]>> On Behalf Of Les Ginsberg (ginsberg)
Sent: Tuesday, February 18, 2020 6:25 PM
To: [e-mail]<mailto:[e-mail]>
Subject: [Lsr] Flow Control Discussion for IS-IS Flooding Speed

Two recent drafts advocate for the use of faster LSP flooding speeds in IS-IS:

https://datatracker.ietf.org/doc/draft-decraene-lsr-isis-flooding-speed/
https://datatracker.ietf.org/doc/draft-ginsberg-lsr-isis-flooding-scale/

There is strong agreement on two key points:

1)Modern networks require much faster flooding speeds than are commonly in use today

2)To deploy faster flooding speeds safely some form of flow control is needed

The key point of contention between the two drafts is how flow control should be implemented.

https://datatracker.ietf.org/doc/draft-decraene-lsr-isis-flooding-speed/ advocates for a receiver based flow control where the receiver advertises in hellos the parameters which indicate the rate/burst size which the receiver is capable of supporting on the interface. Senders are required to limit the rate of LSP transmission on that interface in accordance with the values advertised by the receiver.

https://datatracker.ietf.org/doc/draft-ginsberg-lsr-isis-flooding-scale/  advocates for a transmit based flow control where the transmitter monitors the number of unacknowledged LSPs sent on each interface and implements a backoff algorithm to slow the rate of sending LSPs based on the length of the per interface unacknowledged queue.

While other differences between the two drafts exist, it is fair to say that if agreement could be reached on the form of flow control  then it is likely other issues could be resolved easily.

This email starts the discussion regarding the flow control issue.




_________________________________________________________________________________________________________________________



Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc

pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler

a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,

Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.



This message and its attachments may contain confidential or privileged information that may be protected by law;

they should not be distributed, used or copied without authorisation.

If you have received this email in error, please notify the sender and delete this message and its attachments.

As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.

Thank you.