Re: [lisp] Mirja Kühlewind's Discuss on draft-ietf-lisp-rfc6833bis-13: (with DISCUSS and COMMENT)

["Mirja Kuehlewind (IETF)" <[e-mail]>]

Hi Dino,

catching up from here, because I think I never got a reply from you. Please see below.

> Am 18.09.2018 um 13:05 schrieb Mirja Kuehlewind (IETF) <[e-mail]>:
> 
> Hi Dino,
> 
> please see below.
> 
>> Am 13.09.2018 um 23:46 schrieb Dino Farinacci <[e-mail]>:
>> 
>> I have a new diff file for -15 of 6833bis. I have included changes from your emails and from Colin’s. See my responses inline. I have trimmed a bit of text to hopefully make it more readable for others.
>> 
>> I will only submit the new revision if you agree with the changes and there are no objections from 	the LISP working group. Thanks in advance.
>> 
>>>>> 
>>>> The multiple EID-record encoding is already spec’ed and is in the protocol. And since Map-Requests are triggered by packet arrival or network management tools, they tend to be sent for a single EID. What is for further study is if a ITR would want the entire map-cache, so would it request a set of aggregates and would a replier return all more specifics for the aggregates requested.
>>> 
>>> I guess this sentence should be revised then.
>> 
>> Fixed.
> 
> The sentence still talked about „future version of the protocol“. Is that still necessary? 

The sentence is still there. Please fix.

> 
>> 
>>> 
>>>> 
>>>>> Further given there is no new version, I wonder if the changes as outlined in
>>>>> section 10 are all backward-compatible? Especially for the introduction of the
>>>>> Message-Notify-Ack message, I guess there is no problem if a server sends it,
>>>>> however, as the sender of the Message-Notify message might not know if the
>>>>> other end supports sending of the Message-Notify-Ack it can't rely on it. This
>>>>> should be further discussed in the doc! Or is there another strategy to achieve
>>>>> backward compatibility?
>>>> 
>>>> The Map-Notify-Ack has been there since day-1 in implementations. This is just IETF catching up. And it took over 5 years. So we really don’t have a compatibility situation. And didn’t want to create one in the specifications. So consider it a day 1, version 1 message. It really wasn’t added later.
>>> 
>>> This should be stated in section 18.
>> 
>> Done.
> 
> I would specifically state here that for that reason no interop problems are expected.

This is okay but could be more explicit.

> 
>> 
>>> 
>>>> 
>>>> And big part of this push for standardization is to have the specs “catch up” with the implementations. The implementations are way further ahead than the specs.
>>>> 
>>>>> 2) Size and MTU
>>>>> 
>>>>> As outlined in the TSV-ART review (Thanks Colin!) this document does not
>>>>> discuss fragmentation or Path MTU discovery. RFC8085 recommends to either
>>>> 
>>>>> perform Path MTU discovery or limit the message to 576 bytes for IPv4 or 1280
>>>>> bytes for IPv6 (minus any static header). As this seems to be an appropriate
>>>>> size for LISP messages, I would recommend this approach. Relying on IP
>>>>> fragmentation (as indicated in the reply to the TSV-ART review) is not
>>>>> recommended by RFC8085 as this would lead to IP packet without a UDP header, in
>>>>> the case of LISP, which can cause problem and loss when NATs are involved. In
>>>>> any case the chosen approach needs to be further discussed in the doc.
>>>>> 
>>>> 
>>>> This is a data-plane feature so it is discussed in 6830bis and RFC6830.
>>> 
>>> This is a different issue. The control plane protocol as specified in this doc also needs to consider fragmentation and MTU. Please read RFC8085 accordingly.
>> 
>> I have added text per Colin’s comments.
> 
> Just pointing to RFC8085 is not enough. This spec must specify that messages MUST not exceed the MTU and fragmentation should not be used. And further either recommend to use MTU discovery or just restrict the message size according to the recommendations in RFC8085. I would recommend to just do the later for simplicity.

This is still try. Just point to RFC8085 is not sufficient. RFC8085 gives recommendations. Each protocol spec must require the respective approach that is most appropriate for that protocol to recommend implementing.

> 
>> 
>>>> 
>>>>> 3) Rate-limiting and congestion control
>>>>> 
>>>>> Sec 5.3: "Map-Requests MUST be rate-limited.  It is RECOMMENDED that a Map-
>>>>> Request for the same EID-Prefix be sent no more than once per second."
>>>>> As already noted by the TSV-ART review (Thanks Colin!), RFC8085 actually
>>>>> recommends to not send more the one packet per 3 seconds, and that is a
>>>>> restriction for all traffic not on a per-receiver base, or implement congestion
>>>>> control. This limit is meant to not only protect the receiver but also the
>>>>> network from overloading. Why do you use a smaller interval here? Also if
>>>>> (appropriate) rate limiting is used, this should either be a MUST or more
>>>>> explanation when it is okay to use a smaller rate limit should be provided.
>>>>> However, after all, I don't think you those the right approach here for rate
>>>>> limiting. A Map-Request is always expected to be followed by some reply. For
>>>>> these kind of communication pattern, RFC8085 recommends to limit the number of
>>>>> outstanding requests to 1 (see sec 3.1.1 of RFC8085 recommending one packet per
>>>>> RTT), also for all traffic and not only per receiver. However, this would also
>>>>> require to implement some simple mechanism to detect a message as lost (see
>>>>> also further below in point 4).
>>>> 
>>>> We have referenced RFC8085 in the too be published -14 version of 6833bis.
>>> 
>>> I didn’t find a reference in -14. However a reference is not enough, the specified behavior is not appropriate and needs to change. If you need further help, please consult the respective transport experts, e.g. Gorry Fairhurst who is an author of RFC8085.
>> 
>> I have added text. We have a smaller interval because 3 seconds is way too long (arguably 1 seond is too) to populate a map-cache. Note during that time, packets that arrive to the ITR are discarded. So this occurs if just one Map-Request is lost.
> 
> As I said, just providing a pointer to RFC8085 is not sufficient. 

Again, still true!

> 
>> 
>>> 
>>>> 
>>>>> Similarly I'm not sure about the intent of this requirement in section 5.5:
>>>>> "Map-Replies SHOULD be sent for an EID-Prefix no more often than once
>>>>> per second to the same requesting router. "
>>>>> My understanding is that Replies are only sent when a request is received. Why
>>>>> is this additional rate limit needed? Again if used it should be 3 seconds for
>>>>> all traffic to be inline with RFC8085. Also again, why is that not a MUST?
>>>>> Further recommendation are needed here.
>>>> 
>>>> Because the source is a bad-actor and sending new Map-Requests (with a new nonce).
>>> 
>>> Not sure if that is the right choice because it may interact badly with loss detection. However, as I said the specified behavior need to be refined at various places in the doc.
>> 
>> Well there is not many choices.
> 
> You can choose to not have this rate limiting but limit the total number of request per requesting router per time interval. That would trigger a reply immediately but still limit the total load in case of „bad-actors“.

There is still further guidance missing here.

> 
>> 
>>> 
>>>> 
>>>>> Further section 6.1 say "Both the SMR sender and the Map-Request responder MUST
>>>>> rate-limit
>>>>> these messages.  Rate-limiting can be implemented as a global rate-
>>>>> limiter or one rate-limiter per SMR destination."
>>>>> This seems to be the same rate limit as mention above, or not...? It would
>>>>> probably make sense to rate limit the SMR even further. Please clarify and
>>>>> provide more guidance, e.g. what should the value of a potential additional
>>>>> rate limit for SMR be?
>>>> 
>>>> Some of this machinery is left to the implementor. But we have published some map-cache management guidelines in the lisp-threat RFC.
>>> 
>>> I think clear default guidance must be given in this document. My main problem is that the guidance here is not clear (in relation to other rate limits).
>> 
>> I have tightened it up.
> 
> I don’t think I saw any change in the diff you’ve attached. Please explain what you did.

Still don’t see any changes…

> 
>> 
>>>> 
>>>>> Respectively the following sentence in section 6.1 is also unclear:
>>>>> "The remote ITR MUST rate-limit the Map-Request until it gets a Map-Reply"
>>>>> Why is the rate-limit as currently proposed depend on the fact if a Map-Reply
>>>>> is received? Is the ITR supposed to retransmit the Map-Request…?
>>>> 
>>>> If you are not getting answers to your Map-Requests because of packet loss or MITM, sending them faster is not going to get you a Map-Reply.
>>> 
>>> If you detect loss, you should send slower not faster for congestion collapse avoidance.
>> 
>> That was my point.
> 
> Should should instead specify a similar loss detection and retransmission mechanism for Map-Request as now done for Map-Notify. The rate-limit alone is not helpful.

This is also missing.

> 
>> 
>>> 
>>>> 
>>>>> And finally the Map-Register, Map-Notify and Map-Notify-Ack messages does not
>>>>> seem to have any rate-limits. Recommendations inline with RFC8085 should be
>>>>> provided for the total traffic and not only for a few message types. Again,
>>>>> Map-Notify and Map-Notify-Ack messages should be send only once per RTT as
>>>>> there is a feedback mechanism.
>>>> 
>>>> We have added that in -14.
>>>> 
>>>>> For Map-Register sec 8.2 say: "Map-Register messages are sent periodically from
>>>>> an ETR to a Map-
>>>>> Server with a suggested interval between messages of one minute."
>>>>> However, this a rather a low bound than an upper bound. A required (MUST) rate
>>>>> limit is still needed.
>>>> 
>>>> That is not rate-limiting to avoid message reduction. It is a soft-state way of keeping registration state alive in the map-server(s).
>>> 
>>> That fine but you also need to specify normatively an upper bound.
>> 
>> The recommended upper bound is 1 minute. If you are not getting your point across, please be more specific.
> 
> Then you have to say „MUST NOT send more than one message per minutes“. The current text just gives a recommendation what a go value might be but does not specify an actually limit.

This is still outstanding.

> 
>> 
>>> 
>>>> 
>>>>> 4) Loss detection and retransmission
>>>>> 
>>>>> As also mention by the TSV-ART review (Once more thanks to Colin!), this spec
>>>>> has an ACK mechanism for Map-Requests and now also for Map-Notify, however, it
>>>> 
>>>> And Map-Notify are acks to Map-Registers when requested.
>>>> 
>>>>> does not specify what to do if the ACK is not received (loss detection and
>>>>> retransmission scheduling). This makes the spec incomplete and needs to be
>>>>> further specified in the doc (and also has a relation to the point 3 above of
>>>>> course).
>>>> 
>>>> We focus on why a Map-Request is being generated (to populate the map-cache) and a Map-Reply not only provides information to be put in the map-cache, but acts as an ackl now. If the Map-Request nonce is being replied with a Map-Reply with the same nonce, Map-Requests no longer need to be sent and the requestor is happy with the result (and hence it was reliable).
>>> 
>>> That all fine. However, you don’t specify the behavior if the ack is not received. How/when is the message assumed to be lost? When should it be retransmitted? How often? When to give up? Should exponential backoff be used? This whole part s completely missing in the spec.
>> 
>> Have added text.
> 
> Thanks!
> 
> Not sure I understand the first sentence:
> 
> „A sender of an unsolicited Map-Notify message (one that is not used	
> 	   as an acknowledgment to a Map-Register message) will follow the	
> 	   Congestion Control And Relability Guideline sections of [RFC8085].“
> 
> The rest of the text is fine and gives a good spec. I think it is okay to mention that that spec is inline with RFC8085 but other than that I’m not sure what this first sentence is telling me…?

First sentence is still there and confusing from my point of view.
> 
> 
> And also the last sentence is not fully clear:
> 
> "After this time, the Map-	
> 	   Notify sender can only get the RLOC-set change by later querying the	
> 	   mapping system or by RLOC-probing one of the RLOCs of the existing	
> 	   cached RLOC-set to get the new RLOC-set.“
> 
> I don’t think there should be an attempt to later try again. Probably the system should log or notify an error instead. However, if it makes sense to try later again you have to specify what later means (hours, days?) and how often it should try again before it finally gives up and logs an error.

Can you please further clarify this?

> 
>> 
>>> 
>>>>> 2) I find the security requirements in this doc very unsatisfying. Most
>>>>> important the doc requires the support of authentication mechanism but not the
>>>>> use of it. I would like to see more clear MUST requirements here. Further,
>>>>> today and at this stage of the protocol (moving from exp to PS) I find it not
>>>>> acceptable anymore to have certain security feature as optional and outsourced
>>>>> into a different work-in-process draft. However, I leave further discussion to
>>>>> the SEC ADs.
>>>> 
>>>> Can you cite where this is a problem. Are you saying we are not supplying enough MUST text?
>>> 
>>> In section 8.2 a few SHOULDs are used but no discussion about when it would be appropriate to not follow these SHOULDs. Further these SHOULDs mainly just say what should be implemented and provided, however there is no normative language that these endpoint SHOULD/MUST actually be authenticated or that the configured EID-Prefixes list MUST be enforced.
>>> 
>>>> 
>>>>> Clarification questions:
>>>>> 1) Sec 5.3.:
>>>>> "For the initial case, the destination IP
>>>>> address used for the Map-Request is the data packet's destination
>>>>> address (i.e., the destination EID) that had a mapping cache lookup
>>>>> failure."
>>>>> Does that mean that the Map-Request needs to use the IPv4 or IPv6 depending on
>>>>> the IP version used by the initial message from the EID. Is that always the
>>>>> case or just for this initial message? I would assume that for all other cases
>>>>> this is actually independent...? Because otherwise there would be a constraint
>>>>> on what needs to be requested. I would like t see further clarification about
>>>>> this in the doc.
>>>> 
>>>> No it doesn’t. It depends on the address-family of the map-resolver. And that is configured.
>>> 
>>> Okay, if the map-resolver is IPv6 and I receive an IPv4 packet, how can I copy the "the data packet's destination address“ which is IPv4 in to the "destination IP address used for the Map-Request“ which is IPv6? Or even worse, the other way around.
>> 
>> Because a Map-Request going to the mapping system is “ECM’ed” “Encapsulated Control Message” with the header address-family as the EID being requested.
> 
> You say that if the EID send e.g. an IPv4 message it also has to be encapsulated in IPv4? That is stated nowhere. If that is the intention that has to be specified. However, I don’t really see why this restriction needs to exists…?

This seems still to be missing.

> 
>> 
>>> 
>>>> 
>>>>> 2) In section 5.3: "The ITR MAY include all locally
>>>>> configured Locators in this list or just provide one locator address
>>>>> from each address family it supports."
>>>>> Would it make sense to include a SHOULD requirement to at least the address
>>>>> family that is used to send the Request is included (to increase chance to
>>>>> enable a communication/get a reply)…?
>>>> 
>>>> But all address-families are used all the time. And in some ITRs, usually there is no IPv6 at the edge. 
>>> 
>>> What? You say all address families are used all the time but not in some ITRs…? I don’t understand what you want to say. However, if e.g. an IPv4 packet is received and processed, it probably doesn’t make sense to only include the IPv6 locator address (and not the IPv4 locator address), especially as it is unknown if the ITR support IPV6 (while it has been proven to support IPv4). Currently the spec however would allow that. I’m saying it should to make sure there is no unnecessary failure case her.
>> 
>> RLOCs for an xTR can be a collection of IPv4 and IPv6 addresses. They are tested by remote ITRs to see if they are reachable. When they are, they may be used. When they are not, no packets get encapsulated to those RLOCs.
>> 
>> The Routing Locator Reachability section explains this.
> 
> This is all understood. I’m still wondering if there should be more normative recommendations which locator addresses should be included. 


Still open I guess.

> 
>> 
>>> 
>>>> 
>>>>> 3) Sec 5.4: "If all Weights for a Locator-Set are equal,
>>>>>  the receiver of the Map-Reply will decide how to load-split the
>>>>>  traffic. "
>>>>> Shouldn't the receiver in this case split the traffic equally? Otherwise how
>>>>> would you signal that the traffic should be split equally? Maybe use all zero
>>>>> instead to let the receiver decide…?
>>>> 
>>>> It means the ITR will load-split according to hashing. Versus if the priorities are not equal, it is the ETR deciding how packets ingress the LISP site.
>>> 
>>> This is not what the text says. Please clarify in the doc.
>> 
>> It says it in RFC6830bis since this decision is done by the data-plane.
> 
> I still think the text in this doc is not correct.

This is still seems to be the case.

> 
>> 
>>> 
>>>> 
>>>>> 4) sec 6.1: "When an ITR receives an SMR-based Map-Request for which it does not
>>>>> have a cached mapping for the EID in the SMR message, it may not send
>>>>> an SMR-invoked Map-Request."
>>>>> I guess this should be normative and probably also a MUST NOT or at least
>>>>> SHOULD NOT.
>>>> 
>>>> Yes, I changed to SHOULD NOT. Good catch. Don’t want to make it a MUST NOT because both sides could be mobile and the remote side may need to populate its map-cache. That would allow less packet loss if we keep it SHOULD NOT versus MUST NOT.
>> 
>> While trying to change it, I noticed its already there:
>> 
>> 	<t>When an ITR receives an SMR-based Map-Request for                           
>>        which it does not have a cached mapping for the EID in                         
>>>>>>   the SMR message, it SHOULD NOT send an SMR-invoked                             
>>        Map-Request. This scenario can occur when an ETR sends                         
>>        SMR messages to all Locators in the Locator-Set it has                         
>>        stored in its Map-Cache but the remote ITRs that receive the                   
>>        SMR may not be sending packets to the site. There is no                        
>>        point in updating the ITRs until they need to send, in                         
>>        which case they will send Map-Requests to obtain a                             
>>        Map-Cache entry.</t>
> 
> Good. Thanks! I guess that was changed based on someone else comment already.
> 
>> 
>>>> 
>>>>> 6) Section 8.2 says: "Note that the Map-Notify message
>>>>> is sent to UDP destination port 4342, not to the source port
>>>>> specified in the original Map-Register message."
>>>>> Actually why is that?
>>>> 
>>>> cisco interperability.
>>> 
>>> I would expect to still see more reasoning in the doc.
>> 
>> Sorry, I was wrong in my reply. I read Map-Reply. For Map-Registers this is normal behavior that a request is sent to a well-known port and a reply is sent *from* the well known port.
> 
> Okay, then I still don’t understand WHY this is done?

Would still be nice to add some reasoning.


Mirja


> 
> Mirja
> 
> 
>> 
>> Thanks again for all the effort,
>> Dino
>> 
>> <rfcdiff-6833bis.html>
>> 
>