Re: [Idr] WG Last Call on Extened Message Support

["Susan Hares" <[e-mail]>]

Robert: 

 

Thank you for your message and your concerns regarding BGP in the Internet.   See my comments below. 

 

Sue 

 

From: Robert Raszuk [mailto:[e-mail]] 
Sent: Thursday, February 14, 2019 12:34 PM
To: Susan Hares
Cc: Bruno Decraene; idr@ietf. org
Subject: Re: [Idr] WG Last Call on Extened Message Support

 

Dear Sue & authors of this proposal,

 

>> Therefore, putting an attribute which cannot be decomposed to 

>> 4096 octets or less in an Extended Message is a likely path to 

>> routing failure.

>That's brilliant statement but not enough to proceed with this spec. 

>See the fundamental change that today such issue is in vast majority of cases locally significant. That means that if I send NLRI to BGP it will have a decent chance to be received by all intended parties. 

[Sue]: The reason you receive this NLRI is due to the peer-wise agreements of all the BGP peers in the cloud.  Each peer agrees to send routing based on its policies and capabilities.   Policy and implementation specific details are not in BGP specifications. 

>With new spec we are saying that it may silently fail somewhere in the middle and we do not do anything about it. That's to me showstopper. 

[Sue]: Where do you see the point it must silently fail or an operator will not do anything about it?  

The error message handling specified in the draft and in RFC4271.  If see errors in the error handling as specified please send that to the list.   These errors must be on the peer-to-peer protocol exchange. 

The peer-wise agreements specify the length of a BGP message.  BGP peers receiving a large packet and expecting to send it to a peer which has 4096 bytes as a message size, will need to repackage it.  If it cannot be repackaged, then the BGP implementation needs to warn the operator.   If the BGP peers cannot repackage BGP messages today, there should be error messages and “bells” ringing.  If the implementation does not have the appropriate warning systems, please see your vendor.

What show stopper deployed today are you talking about?  I do not perceive any show-stopper on the peer-to-peer communication. 

At min if given NLRI can not be sent with XXL attributes to some peers (say on the RR which Bruno mentioned) we must specify protocol extension in this very spec on how to deal with it. 

[Sue]:  We cannot send the 12K large community attribute now.  This protocol specification deals with what happens when you send and receive BGP packets.  The BGP peers negotiate the length of the BGP message that can be sent/received.  It has been tested and deployed (see comment about OpenDaylight) for some years.  

Your description indicates a “network-wide” requirement.  A network-wide requirement is based on the specific needs of an application + BGP.  For example, some Data center BGP applications are very different than EVPN BGP over MPLS.  

Such extension may be as simple as to define a short attribute BGP_PIPE_TOO_THIN which will be inserted in the place where XXL attribute just did not fit smaller message size peers to tell them then "Hey something is broken with this NLRI" - pls check. Then it will be up to local policy to install such NLRI or drop it. 

[sue]This idea is orthogonal to the basic BGP engine.  It is similar to ideas that have been kicked around in Grow and IDR for a while.  If this use case gives new focus to these ideas, please write a specification and suggest it to the IDR and Grow WG.   IDR can look at the passage of error messages.  Grow can look at how useful it is for the general BGP.  We’ve talk about the error message concept for 10+ years, but the tradeoff for the error message has not given enough benefits to operators.    Perhaps this is the “killer-application” application..  

Thx,
R.

 

 

On Thu, Feb 14, 2019 at 6:21 PM Susan Hares <[e-mail]> wrote:

Bruno: 

 

Thank you for careful considerations on the use case.   I think we have translation issues between our “English” language again so I have tried to be precise in the detailed comments.  I’ve summarized my comments as well to help us take the next step. 

 

Sue 

 

Summary

----

Good catch on the 1000 large BGP communities blowing up the message size!  This is a case where a code path could get caught with program error.  The error handling (section 6.1 in RFC4271 per section 5 in the extended specification) IMHO handles the case where the attribute is longer than the message.   Please let me know if I’ve missed something.   

 

As I understand it,  you are asking this specification to be expanded to add notes on policies on what implementation should do to correctly combine RFC4271 length sizes with extended length sizes.   The implementers and operations people commented earlier that this should not be in the protocol specification other than as a brief footnote.  

 

I agree there is a need for such standardization as BCPs for particular use cases (large communities, BESS (EVPN or centralized controller), SPRING, BGP-LS information (see Daylight comments yesterday), SD-WANs (rtgwg), and SPRING.   I think the chairs of the WGs utilizing should consider if there is need for standardization for a minimal set of policy for message sizes for specific use cases.   I will be glad to send request to these WG or to host something in IDR if there is policy that can be used across all use cases. 

 

To provide a slight expansion to point people in this direction, how about this change to paragraph in section 4: 

 

Old/

   A BGP announcement will, in the normal case, propagate throughout the

   BGP speaking Internet; and there will undoubtedly be BGP speakers

   which do not have the Extended Message capability.  Therefore,

   putting an attribute which can not be decomposed to 4096 octets or

   less in an Extended Message is a likely path to routing failure.

/

New/

   A BGP announcement will, in the normal case, propagate throughout the

   BGP speaking Internet; and there will undoubtedly be BGP speakers

   which do not have the Extended Message capability.  Therefore,

   putting an attribute which cannot be decomposed to 4096 octets or

   less in an Extended Message is a likely path to routing failure.

   One example of a valid attribute which cannot be precisely decomposed if it 

   sent in Extended message is a large community attribute with 1000 communities. 

   (12,000 bytes).  

/

[PS – in American English “Can not” is normally concatenated “cannot”.] 

   

 

 

 

From: [e-mail] [mailto:[e-mail]] 
Sent: Thursday, February 14, 2019 3:29 AM
To: Susan Hares
Cc: [e-mail]; 'Jakob Heitz (jheitz)'
Subject: RE: [Idr] WG Last Call on Extened Message Support

 

Sue,

 

Thanks for the reply. Please see inline [Bruno]

 

From: Susan Hares [mailto:[e-mail]] 
Sent: Wednesday, February 13, 2019 8:03 PM
To: DECRAENE Bruno TGI/OLN
Cc: [e-mail]; 'Jakob Heitz (jheitz)'
Subject: RE: [Idr] WG Last Call on Extened Message Support

 

Bruno: 

 

I’d like to answer some of the questions you sent Randy based on comments (public and private) the chairs and I as the shepherd received on this draft in our last WG LCs.   

 

Several of the comments you questioned were put in to answer other questions.   This gets into a catch-22… problem – where you put changes in for one person/take out the changes out for another.  In academic circles this is called the reviewer #2 problem (reviewer 1 insists changes go in, reviewer 2 insists changes go out) during publication.  Therefore, I’m I am going to answer most of your questions where this is the case.  

 

I believe that Randy has new version with the changes I’ve indicated.  I’m hoping he’ll just post that version.  AFAILK, we do have 2 compliant implementations. 

 

Sue 

 

 

 

<shepherd hat on> 

This draft’s technology provides efficient processing for some application by allow a huge BGP message size.  Whether “chunking” BGP to 4096 byte messages or packing large amounts of data into 65,000 byte message is more effective for a particular application is something only an implementer of a particular BGP implementation  can decide.   This draft simply gives freedom for applications to not be restrict to an artificially small message size.  The market place will choose whether this technology is used or not. 

 

As Jakob pointed out, changing a highly tuned BGP implementation which “chunks” BGP data into 4096 byte messages may involve substantial rework of the implementation.  

 

 

(snip) 

From: Idr <[e-mail]> On Behalf Of [e-mail]
Sent: 29 January 2019 13:33
To: Susan Hares <[e-mail]>
Cc: [e-mail]
Subject: Re: [Idr] WG Last Call on Extened Message Support

 

Hi WG,

 

>Please find below some comments.

>As of today, I don’t believe this specification is ready to be progressed to IESG/RFC, especially for a document >updating RFC 4271 (core BGP spec).

 

> The WG chairs intend to forward this draft to the IESG with the current level of implementation.  

 


>https://trac.ietf.org/trac/idr/wiki/draft-ietf-idr-bgp-extended- <https://trac.ietf.org/trac/idr/wiki/draft-ietf-idr-bgp-extended-%3eimplementations> >implementations says : 5a 

Does not send Extended Message capability 

Yes 

Yes 

Yes 

 

>I may be misunderstanding the implementation report, but my reading of the above is that none of the reported >implementations sends the capability hence no implementation supports draft-ietf-idr-bgp-extended-messages.. 

>Here this document is updating RFC 4271, so it is not a minor extension for a niche use case. So I don’t see the >arguments for not requiring the IDR’s usual two interoperable implementations.

 

>In our mail thread, we discovered that the implementation report should have indicate

>full support.   I’m waiting for Olivier to update based on the last point, but my understanding is that 

All implementations have full support.  

 

[Bruno] ok, solved

>----

>§ 1

>“ As BGP is extended to support newer AFI/SAFIs and

>   newer capabilities (e.g., [ <https://tools.ietf.org/html/draft-ietf-idr-bgp-extended-messages-27#ref-I-D.ietf-sidr-bgpsec-protocol> I-D.ietf-sidr-bgpsec-protocol]), there is

>   a need to extend the maximum message size beyond 4096 octets.  “

> 

> https://tools.ietf.org/html/draft-ietf-idr-bgp-extended-messages-27#section-1

 

 

>[ <https://tools.ietf.org/html/draft-ietf-idr-bgp-extended-messages-27#ref-I-D.ietf-sidr-bgpsec-protocol> I-D.ietf-sidr-bgpsec-protocol is now RFC 8205 (thanks for updating the reference). It has removed the normative/any >reference to draft-ietf-idr-bgp-extended-messages. So presumably BGP Sec does not need draft-ietf-idr-bgp->extended-messages. Can we have an update on this?  Can the introduction of draft-ietf-idr-bgp-extended-messages >be updated to introduce on the real reasons/needs?

 

SIDR gave up waiting on IDR’s work and took a reduced size of encryption.   The security people were very sad about the reduction.  How is this working in practice? WG sidrops can answer this question.  

 

The real problem is there is finite message space.  

[Bruno] So too bad the draft does not solve the real problem ;-) (as message space is still limited)

 

[Sue]: The message is still finite (smile),  but I hope 65,635 (smile) is probably enough for 10 years.   

 

There are two ways to send lots of BGP routes and/or attributes – to “chunk” these into small message sizes or place these into a large message.    Some applications may always want to “chunk” into message sizes smaller than 4096.   Some application make work more efficiently utilizing large message size.  As the potential number of NLRI and BGP attributes grows, new applications may consider large message sizes to a more reasonable way to exchange amounts of BGP data.    

 

The decision on how to send BGP messages over the wire is a peer-by-peer decision in a network.   Each BGP peer is in charge of reformatting the BGP message it sends to the remote BGP peer.   The BGP capabilities in this draft represent an agreement between two BGP peers.  These are facts of basic BGP implementations

[Bruno] Let’s take the example of a BGP Route Reflector having a mixed of clients, some supporting Extend Message (EM) and some not.

>From an EM client, it receives a BGP update containing one (single) NLRI with an Total Path Attribute Length bigger than  4090 byte (e.g. having 1000 large communitiess)

Can you explain me how it behaves to send this NLRI to an IBGP peer not supporting EM?

[Sue]:  Gladly… 

A BGP route reflector (RR-A) is peering with each of its BGP clients.   It agrees with each of its clients (BGP-C1, BGPC2, BGPC3) whether the BGP Extended messages is accepted or not (based on capability announcement). 

Let us assume that BGP-C1 does not support extended messages, and  BGP-C2 and BGP-C3 do support extended messages. 

 

If the policy in RR accepts a total path length of 1000 large communities with  a single NLRI, it may send the BGP attribute via extended communities to BGP-C2 and BGP-C2.

The policy in the RR must specify whether: 

a)      the “too long” attribute path length must be dropped, or 

b)      the BGP attribute path can be truncated. 

 

This is a policy issue within the BGP peer and the network operation staff.  Each operator may choose to write this policy differently. 

 

Rather than specify the numerous cases in the protocol specification,  the previous discussions (private and public) asked us to trim this information out of the protocol specification. 

 

If you believe this is worthy of a use case draft for the sake of BESS, LSVR, SPRING, Grow and others, I will happily WG chair sponsor a draft that can be reviewed by all these WGs. 

 

 

– so I do not believe adding these facts to Randy’s draft is useful.  In fact, in the previous rounds of review this point was made. 

 

Your question (and perhaps Jakob’s) is whether we have reached a time/place where the “chunking” approach will fail on a peer-by-peer basis?   This draft does not make this statement.  

 

The point of this draft is to enable more efficient processing by increasing the message size for BGP peers that send a lot of data (e.g. BGP-LS) .   The choice to “chunk” or stream in large BGP message is an implementation choice for a particular BGP application.  

 

----

§4

§3 says “A peer which does not advertise this capability MUST NOT send BGP

   Extended Messages, and BGP Extended Messages MUST NOT be sent to it.”

[I believe randy accepted this change with the following text in version 4 [upcoming below]]/. 

 

--------------

 

Fine. Text in §4 should probably be aligned with the above ..e.g.

OLD: A BGP speaker

   MAY send Extended Messages to its peer only if it has received the

   Extended Message Capability from that peer.

 

NEW:

A BGP speaker

   MAY send Extended Messages to its peer only if it has sent and received the

   Extended Message Capability to and from that peer.

 

 

----

>“   Applications generating information which might be encapsulated

>   within BGP messages MUST limit the size of their payload to take the

>   maximum message size into account.”

> 

>I don’t see what new behavior is been defined here. If there is none, I would suggest to remove this sentence

 

Bruno – people asked to put this text in as an operational warning.  At this point, I’m going to say unless there is harm it should remain.  

[Bruno] OK.

 

----

>   A BGP announcement will, in the normal case, propagate throughout the

>   BGP speaking Internet; and there will undoubtedly be BGP speakers

>   which do not have the Extended Message capability.  Therefore,

>   putting an attribute which can not be decomposed to 4096 octets or

>   less in an Extended Message is a likely path to routing failure.

> 

>The issue is not specific to attributes bigger than 4096 octets, but to BGP message whose length is bigger than >4096, irrespective of the size of each attribute.

>Please elaborate on what you mean by 

> 

>“an attribute which cannot be decomposed to 4096 octets”

 

An attribute which cannot be decomposed to 4096 is an attribute with length 6000 inside of a BGP message which requires the entire attribute to be included.   No such attribute exists today.   If a small BGP peers create a “killer application” that uses a 6000 byte attribute, the rest of the Internet does not have to use it.   However, the person defining that attribute must carefully specify it must go with Extended BGP messages. 

 

 

[Bruno] I don’t think that I agree with this. E.g. I don’t see where RFCs 1997 or 8092 [(large)community] limits the size of their attribute. Hence I don’t see why you could not have, today (with Extended Message support) , attributes larger than 4096 octets. Most probably same status for the BGPSec attribute since they wanted for more space.

[Sue]: You make a good point about the possibility of large communities not limiting the size.  It is worthy of a change to both the large communities and a footnote in Randy’s specification.  How about the addition of a single sentence to point people in the right direction. 

 

Old/

   A BGP announcement will, in the normal case, propagate throughout the

   BGP speaking Internet; and there will undoubtedly be BGP speakers

   which do not have the Extended Message capability.  Therefore,

   putting an attribute which can not be decomposed to 4096 octets or

   less in an Extended Message is a likely path to routing failure.

/

New/

   A BGP announcement will, in the normal case, propagate throughout the

   BGP speaking Internet; and there will undoubtedly be BGP speakers

   which do not have the Extended Message capability.  Therefore,

   putting an attribute which cannot be decomposed to 4096 octets or

   less in an Extended Message is a likely path to routing failure.

   One example of a valid attribute which cannot be precisely decomposed if it 

   sent in Extended message is a large community attribute with 1000 communities. 

   (12,000 bytes).  

/

  

 

This text was added during an earlier discussion where the error cases of a large BGP attribute was given.  The originator of the common could not provide the exact description, but we agreed to put this  “warning label” into the draft.   This is one of the cases where adding/removing text gets to be problematic. 

 

>---

>“   It is RECOMMENDED that BGP protocol developers and implementers are

>   conservative in their application and use of Extended Messages.”

> 

>What does this mean exactly? That they don’t use this extension? 

>That they don’t use this extension unless XX_TO BE SPECIFIED_XX?

 

This warning was text that other participants asked to be added. 

 

It is a common sense warning.  Extended messages are enabled on a peer by peer basis as all BGP capabilities are.  If you run into a Peer that does not support the larger sizes, the BGP messages will need to be reformatted to the smaller size.   Running immediately to the Extended messages as the default could cause problems. 

 

[Bruno] I think that the sentence could be elaborated. Indicating the problem, how the specification partially address it, and how the specification does not address it.

In “Running immediately to Extended Message” I don’t think they that the issue is with “immediately”. Whenever you start the transition, your likely to run into problems.

 

 

[Sue]: IMHO specification specifies how to encode the bytes on the wire and the appropriate error handling of the messages.   If I have missed something in this case, please let me know. 

 

The specification does not specify the necessary policies implementation should have in order to decide what to do in the case of mixed network.   The clear indication was this type was not appropriate for the protocol specification.  

 

Earlier discussion indicated that these is policy and mechanism specific.  It is worthy of “use case or application notes” written for specific applications (BESS, SPRING, GROW), but not in the specification.  Again, if it is significant enough concern for operators and implementers – it would be worthy to have combined use case document besides this point.    I’m happy to sponsor cross-area work on this use case. 

 

---

  Future protocol specifications will need to describe how to handle

   peers which can only accommodate 4096 octet messages.

 

>Why is this limited to future specifications? A priori, using existing BGP mechanism (AFI/SAFI, attributes, * >communities) one could exceed the size of 4096 octets. How does the BGP speaker supposed to behave in this ?>case? This should be described in this specification. Note that this is not a case of error handling, as every BGP 

> speaker is behaving as specified.

 

I fail to see your logic.  At this point, all BGP data (AFI/SAFI and attributes) must fit within a 4096 message.   Therefore, a BGP peer A which connects on link 1 to BGP peer B with 4096 message and on link to a BGP peer C with extended messages only has to worry about reformatting the data.  

 

In the future, the first group that defines attribute or an NLRI that does not fit within a 4096 BGP message and cannot be split, will need to define how that works in the specification.  There is no need to refine the current set of specifications.   

[Bruno]

1) cf above, I believe that there are existing BGP Attributes which may not fit into a bytes 4096 BGP message

2)Even in the absence of such attribute, you have the issue of 2 smaller attributes whose cumulative size does not fit into a 4096 bytes message.

 

The problem is introduced by this specification, hence I believe this specification needs to specify how to handle this case.

 

[Sue]:  IHMO the message transmission and reception error handling issues are complete in the combination of RFC4271 and the extended message section 5. 

 

What is not in this specification is the policy issues about what happens if you combine 2 smaller attributes.  For example, if you have a large community attribute of 400 communities (4800 bytes), you will likely run into problems when combining with some longer AS path lengths.  [4800à 5096 bytes is a short jump]. People on the list and in-person argued this was policy and implementation specific details.  

 

I agree with you that it is extremely useful to the operator community to have this type of policy details discussed to see if there is some desire to have common policies and error communities.   It is the type of thing that GROW and IDR should be deeply involved for large communities.   BESS, LSVR, SPRING and others should consider what policy should be adapted for their applications. 

 

>----

>Depending on the above specification, a section describing the operational consequences in a network (such as the >Internet, BGP Enabled ServiceS/VPN networks) is probably needed. Possible consequences could be BGP NLRI >being removed in the middle of such network, or (extended) community (such as Route Targets) been removed. >Both having significant consequences on the availability provided by the network.

 

At this point, there are no attributes or NLRI groupings that do not fit within 4096.   If you define one (as SIDR did), then you will need to have it reviewed by IDR, BESS, SPRING, and operational groups (GROW, etc). 

[Bruno] no, cf above.

[Sue]:  Thank you for reminding me of the potential of large communities to blow up BGP.  I have a new worry to add to my BGP concerns list! 

 

Thanks,

--Bruno

 

 

---

>§4

>OLD: The Extended Message Capability only applies to all messages except for the >OPEN message. 

>Probably

>NEW: The Extended Message Capability applies to all message types except for the >OPEN message (type 1). 

>----

>§8

>“This extension to BGP does not change BGP's underlying security issues »
>Before evaluating this, I think this document should first specified how a BGP 
>messages bigger than 4096 octets is handled when it needs to be sent to a 
>received not supporting this extension.

 

Nits:

>OLD : to reduce compexity

>NEW : to reduce complexity

 

Thanks,

--Bruno

 

From: Idr [mailto:[e-mail]] On Behalf Of Susan Hares
Sent: Tuesday, January 29, 2019 12:33 PM
To: [e-mail]
Subject: [Idr] WG Last Call on Extened Message Support

 

 

This begins a 2 week WG LC on Extended Message Support for BGP (draft-ietf-idr-bgp-extended-messages-27).  You can access the draft at: 

 

 <https://datatracker.ietf.org/doc/draft-ietf-idr-bgp-extended-messages/> https://datatracker.ietf.org/doc/draft-ietf-idr-bgp-extended-messages/

 

The authors should indicate whether they know of any IPR.   Implementers are encouraged to update the  implementation data at: 

 

 <https://trac.ietf.org/trac/idr/wiki/draft-ietf-idr-bgp-extended-implementations> https://trac.ietf.org/trac/idr/wiki/draft-ietf-idr-bgp-extended-implementations

 

The draft provides a means for expanding the BGP message to 65535 octets for all messages except OPEN messages.  BGP message space is running short for all of the potential attributes or additions proposed by BGP-LS features.  

 

The WG chairs intend to forward this draft to the IESG with the current level of implementation.  

 

As you comment on the draft, please consider if: a) the technology is mature, b) the additional space in a BGP message would be helpful for those deploying BGP-LS or SR, and c) if the specification is ready for publication.  

 

Sue Hares (WG Chair, Shepherd) 

 

 

_________________________________________________________________________________________________________________________
 
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
 
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
_________________________________________________________________________________________________________________________
 
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
 
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
_________________________________________________________________________________________________________________________
 
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
 
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

_______________________________________________
Idr mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/idr