Re: [lisp] LISP NAT Traversal [Was: Virtual meeting]

[Dino Farinacci <[e-mail]>]

> Thanks Dino for your reply. Answers inline

No prob.

> 
> On 31/3/20 1:27, Dino Farinacci wrote:
>>> I would also like to add the NAT Traversal draft discussion into the agenda.
>>> 
>> I agree it should on the agenda and an over-due work item we need to get working.
>> 
>> 
>>> We have experience implementing the NAT traversal functionality in our LISP open source implementation (OOR) which have support for Android and IOS devices. We believe that NAT traversal is a critical point for LISP-MN and we would like to share our experience on that. .
>>> 
>>> Our implementation is based on the draft-ermagan-lisp-nat-traversal and during its study some questions have emerged that could be interesting to discus:
>>> 
>> I have an implementation of NAT-traversal in the lispers.net
>>  implementation which is a subset and simpler version of the draft-ermagan-lisp-nat-traversal. I think that NAT-traversal is so important as a core functionality for LISP because many people use it today just to traverse NATs.
>> 
>> I would suggest we make this a working group document. It is in the charter and is long over-due to move from individual submission.
>> 
>> 
>>> 	• What happens when a device handovers between NAT and not NAT interfaces? The xTR has to notify the remote ITRs, however an xTR behind NAT usually only has a default map cache entry with the RTR as a default gateway. Therefore, the xTR cannot notify directly ITRs of the change of the mapping.
>>> 
>> If the remote ITR subscribes for 0.0.0.0/0 then any more specific mappings which are registered and Map-Notified by the Map-Server to the subscribers. That is one way the problem can be solved. 
> So here, you are suggesting to use the publish/subscribe draft isn’t it? Yes this could be a good solution. Is there any study of scalability regarding this option?

Yes, that is what I’m suggesting.

>> Another way is now that the xTR is not behind a NAT, it can SMR currently RLOCs it is talking to. Which also may be an RTR that the remote ITRs can be using (when they are behind NATs). That causes the non-NAT ITR and the remote RTR to do mapping lookup to get the global/non-translated RLOC of this xTR.
> When the xTR is only using the RTR (I believe is the usual case), xTR doesn’t know who are its remote ITRs to send and SMR so it depends on how the RTR react to SMR.

When the remote ETR isn’t behind NAT, the RTR can reach it directly but so can the ITR. 

>>> 	• What happens in the previous case if the not NAT interface is IPv6? IPv6 is a ­special case. We want to maintain the connectivity with established connections (keep using the same RTR, which has the map-cache). For that to work, the RTR needs to have an IPv6 to receive an SMR from the xTR after the handover (from the new xTR’s IPv6 RLOC). Therefore RTRs announced by the Map Server should have both IPv4 and IPv6. As we don't know which RTR IPv6 addresses map to which IPv4 addresses, all RTRs should be notified.
>>> 
>> It can be a local decision if an ITR uses an RTR for IPv6 traffic or just encapsulate directly to the ETRs that register the EID. Most customers are going to want the shortest path so I have seen the latter be deployed.
>> 
>> I would suggest you only use RTRs for IPv6 when you are either doing (1) signal-free IPv6 multicast or (2) have a traffic-engineering requirement. Both in general, specify that IPv4 and IPv6 can and should run ships-in-the-night.
>> 
> What I was referring with this question is , If we are behind NAT using an RTR as a default gateway and then we handover to an IPv6 connection (at that point we are not considering the publish/subscribe solution), If we don't want to lose established connections, xTR has to notify the RTR (SMR) of the change of the mapping. To notify it,

Yes. But now I’m not clear if you the “IPv6 connection” is referring to an IPv6 EID or RLOC. Please clarify.

> the RTR should have an IPv6 RLOC that should be included in the list of RTRs of the Info Reply message.  Depending on what the RTR does with the SMR,  remote ITRs could move stablished connections to direct path or continue using the RTR.

If IPv6 is an RLOC, it is assumed anyone can reach it and a site xTR need not encap packets to an RTR. RTRs for NAT-traversal are used only for IPv4 RLOCs, regardless if the EID is IPv4 or IPv6.

>>> 	• Which should be the procedure of an RTR in front of receiving an SMR from the xTR that just handover from NAT to not NAT? Probably the answer is that the xTR sends an SMR to the RTR which then has two options 1) the 
>>> 
>>> RTR can do an SMR to the appropriates ITRs based on the map-cache entries (but then how to know which of the map-cache entries were used by the xTR) or 2) it can act as a PxTR for the xTR that performed the handover. In that case, until when? Until the expiration of the TTL? Furthermore, what happens when an xTR roams again before the previous TTL expires.
>>> 
>> This is no different than any other type of RLOC change. And we have many mechanisms documented for this. They are:
>> TTL-expiry, SMRs, pubsub Map-Notify from mappingn system.
>> 
> TTL-expiry is not realistic for mobility scenarios where handovers can be frequent and you don’t know when will happen

Well you will find with experience that link-layer assocaition estalishment takes seconds. So having the map-cache TTL around 3-5 seconds is reasonable. Follow the numbered flows.

> The behaviour of an RTR receiving an SMR is what I would like to clarify with this question
> 
> 
> pubsub is a good option but at this point I would like to clarify what happens when an RTR receive an SMR.

Note, that SMRs will cause a flurry of messages depending on how aggressive the ETR wants the ITRs sending to it to be updated. And requires queuing demands on the ETR for reliable delivery. Hence why data-driven SMRs were introduced in implementations. But that only happens for EID-mobility (EID is not co-located with RLOC) and not in LISP-MN (where EID and RLOC are co-located).

>>> 	• Which is the mechanism to change the RTR used by and xTR?
>>> 
>> Note the xTR gets periodic Info-Replies back from the map-server. The list of RTRs can change. So when the list of RTRs is changed, it is configured in the map-server and the next Info-Request that the NAT-assisted xTR sends is returned with the new list. The lispers.net implementation updates RTRs when the list changes and RLOC-probes each one to make sure they have reachability before using.
> I believe this answer is replying  next question. With this question, my intention was to make constance that the draft doesn’t specify (may be it is not needed to go into this detail level) a mechanism, from the xTR point of view, to use a different RTR from the list of the info reply (for example, due to a better latency time)

We should fix that in the spec then. Thanks for finding it.

>>> 	• Is there any procedure from MS point of view to notify associated xTRs to change the RTR they are using?
>>> 
>> Yes, in the Info-Reply messages the map-sever send, that is in draft-ermagan-lisp-nat-traversal.
>> 
>> 
>>> 	• If a NATed device wants to manage their map request and map replies, which ITR RLOCS should use to fill the Map Request?
>>> 
>> It needs to be reachable RLOCs. And in the lispers.net implementation, an xTR behind a NAT registers it’s translated RLOC and the RTR RLOCs. That is so the remote RTR can use the translated RLOC (and the translated port it learns from Info-Requests) and remote ITRs to use the RTR’s RLOC (since that RTR is the only node that can get packets through the NAT).
> I don’t have the full details of the lispers implementation of NAT-Traversal solution. Is any document describing it?.

I don’t have something written up but I do have slides that show protocol flow. See at the end of this email. It can all be explained in one slide. ;-)

> My concern here is regarding section 5.1.2 of the draft (Map-Request and Map-Reply handling). When we sent an Encap Map-Request to get a mapping, we add a list of iTR addresses to receive the Map-Reply. We can add here the global address of the xTR but if the NAT has not been previously punched, we will not receive never a Map-Reply, at least for symetrics NATs.

You MUST put public-RLOCs in the mapping and the only node that can do that is the ETR. And it discovers its public-RLOC but receiving a Info-Reply from the MS that echos the outer source address and source UDP port from the Info-Request message. The message that the NAT translated the source address and source port.

> 
>> Another topic but related to NAT-traversal.
>> 
>> Note there is a problem I came across during deployment of an IoT use-case. This is a description of the problem with an example. Say I am xTR1 and learn from the Map-Server RTRs A, B, and C. and I want to encapsulate packets for an EID behind xTR2. The encapsulation algorithms indicate that xTR1 use a 5-tuple hash to load-split packets across A, B, and C. And xTR1 knows that A, B, and C are reachable via RLOC-probing. However, xTR1 has no idea if any of A, B, or C, can reach xTR2. So xTR1 can choose an RTR that would drop packets if it couldn’t reach xTR2.
>> 
>> I solved this in the 
>> lispers.net
>>  implementation to have the ETRs only register the RLOC addresses of RTRs if they are currently reachable via RLOC-probing. This is how it works using the example from above:
>> 
>> (1) xTR2 learns of RTRs A, B, and C from the map-server.
>> (2) It uses those RTRs as the RLOCs for the map-cache entry 0.0.0.0/0.
>> (3) It RLOC-probes A, B, and C.
>> (4) It registers its translated RLOC and RTRs A, B, and C in 4 RLOC-records of the Map-Register.
>> (5) That means any remote ITR (xTR1) would encap to either A, B, or C.
>> (6) If xTR2 lost reachability to A, then what is registered is only B and C (and the translated RLOC).
>> (7) Then xTR1 only has a choice of B and C to encapsulate to for the EID behind xTR2.
>> 
>> What limits the solution is asymmetry. Meaning if the path from xTR2 to RTR A is down but the path from RTR A to xTR2 is up, xTR1 would not use RTR A. So bidirectional connectivity is required (for the pair of RLOCs that are part of the RLOC-probe).
>> 
> In the scenario you are proposing, both xTR are behind NAT using the same MS or at least the same RTRs isn’t it?

Well as a LISP-MN roams, it uses the same Map-Server no matter where it attaches to the network. Thereby using the same RTR-list for what the Map-Server is configured with. The Map-Server with policy COULD return a different RTR-list based on the LISP-MN RLOC address or GPS coordinates it registers with.

> What would happen if the xTR 2 is not behind NAT  or it is behind NAT but using different RTRs?

I don’t have it implemented for a multiple sets of RTR-lists. 

NAT-traversal, doing it dynamically and seamless to the user and network administrator is one of the toughest problems I had to solve in my career. It is so messy and has too many combinations that need to work.

Also note, there is a requirement for an xTR to have two interfaces, each connected to different NATs. And load-splitting is required across the two intefaces (inbound)

Dino

> Regards,
> 
> Albert L.
>