Re: [lisp] draft-kouvelas-lisp-map-server-reliable-transport working group adoption

"Joel M. Halpern" <jmh@joelhalpern.com> Wed, 06 December 2017 20:17 UTC

Return-Path: <jmh@joelhalpern.com>
X-Original-To: lisp@ietfa.amsl.com
Delivered-To: lisp@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C399A12426E for <lisp@ietfa.amsl.com>; Wed, 6 Dec 2017 12:17:56 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.701
X-Spam-Level:
X-Spam-Status: No, score=-2.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=joelhalpern.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TxwSFWatjzRM for <lisp@ietfa.amsl.com>; Wed, 6 Dec 2017 12:17:54 -0800 (PST)
Received: from maila2.tigertech.net (maila2.tigertech.net [208.80.4.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BAAB4124207 for <lisp@ietf.org>; Wed, 6 Dec 2017 12:17:54 -0800 (PST)
Received: from localhost (localhost [127.0.0.1]) by maila2.tigertech.net (Postfix) with ESMTP id 9DCE6AE01EF; Wed, 6 Dec 2017 12:17:54 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelhalpern.com; s=1.tigertech; t=1512591474; bh=DdAOQW98y9jWRhYpC7qDDAtGmTmRztUy+rxz4yrDBS4=; h=Subject:To:Cc:References:From:Date:In-Reply-To:From; b=noaMa0rpt1w0y9X95n8EZ4pYh8/e953i9RhZxn9NFdjiJ46SoNGv2/FY1WhwBiCo9 ytCf+8BFkBux4LXKBpbOMhmPKTvOdXx/Q9qEGdUsvT4ZQdr3APc3CotXuhFXb6yJKg /kFr7hWFETrMhLWYfv2IEkdTU2dMgfzo3x26Ah/c=
X-Virus-Scanned: Debian amavisd-new at maila2.tigertech.net
Received: from Joels-MacBook-Pro.local (unknown [50.225.209.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by maila2.tigertech.net (Postfix) with ESMTPSA id E3EED240A33; Wed, 6 Dec 2017 12:17:53 -0800 (PST)
To: "Johnson Leong (joleong)" <joleong@cisco.com>, Dino Farinacci <farinacci@gmail.com>
Cc: "lisp@ietf.org" <lisp@ietf.org>
References: <42781EBC-3E60-425C-BFEC-B13686C654C6@kouvelas.net> <103862EC-37E4-4D43-B5E4-DD187C42FFC2@gmail.com> <6000ffde-430c-7f8c-d0be-9de6077fa6df@cisco.com> <1B364743-5EA6-4261-973B-EECD0DE395D9@gmail.com> <D93B8F35-3F64-4800-B7C0-46715CE1E28E@cisco.com> <56870BB6-1CAB-4389-9680-7F0F7749DD84@cisco.com> <43F14534-07CE-46A1-B682-1A6C170C5507@gmail.com> <AFD152F8-1468-4C04-B4AD-6EF3998D9408@cisco.com> <C9E41BA9-BFBB-4752-BBE7-8F2D7655A9C5@gmail.com> <00086A69-B932-41A2-AC15-2EFB0D6A44E4@cisco.com>
From: "Joel M. Halpern" <jmh@joelhalpern.com>
Message-ID: <52034017-790a-d040-c608-4004a6747dcd@joelhalpern.com>
Date: Wed, 06 Dec 2017 15:17:52 -0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.5.0
MIME-Version: 1.0
In-Reply-To: <00086A69-B932-41A2-AC15-2EFB0D6A44E4@cisco.com>
Content-Type: text/plain; charset="utf-8"; format="flowed"
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/lisp/MsLsIaV1mosAhwoZJjU-7gkEEcY>
Subject: Re: [lisp] draft-kouvelas-lisp-map-server-reliable-transport working group adoption
X-BeenThere: lisp@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: List for the discussion of the Locator/ID Separation Protocol <lisp.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/lisp>, <mailto:lisp-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/lisp/>
List-Post: <mailto:lisp@ietf.org>
List-Help: <mailto:lisp-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/lisp>, <mailto:lisp-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 06 Dec 2017 20:17:57 -0000

<speaking without hats.>
Johnson, I think your example may cause more problems.
I am not sure what you mean by "over-subscribed".
But if the xTR sends a registration over TCP, and gets no answer, it is 
going to asume that it is properly registerd.  If the registration has 
not been received due to the receiver not reading his TCP queue (because 
he is over-subscribed) that is a bad result.

Yours,
Joel

On 12/6/17 2:44 PM, Johnson Leong (joleong) wrote:
> Dino,
> 
> What we're trying to convey is that by using TCP it allows us to achieve reliability which significantly simplifies the code and error handling.  One can achieve the same with UDP with ack and retry but it would be reimplementing what TCP already offers and this added complexity typically makes the code error prone.
> 
> Another benefit with TCP is congestion control, with UDP if we send a map-register and we don't get an ack then we retry.  What if the MS is fully subscribed then this can lead to constant retry.  You can impose backoff mechanism but ultimately this would affect convergence.
> 
> -Johnson
> 
>> On Dec 6, 2017, at 11:17 AM, Dino Farinacci <farinacci@gmail.com> wrote:
>>
>>> Dino,
>>>
>>>>   LISP (the application), does not know that itself, the xTR, is in sync with the map-server. The packets can be in flight or being retransmitted due to loss. But if a Map-Register is sent with a nonce and no Map-Notify is returned the xTR knows for sure the two are in sync.
>>>
>>> An application (and LISP in this case) should always be able to know the state of a (TCP) socket that it has opened with a server. I’m not entirely sure why we would not want to use this information.
>>
>> All an app knows is the socket it has opened and any port it has bind()’ed to. It doesn’t know the connection state in terms of what packets have been sent and what has been ack’ed. There IS NOT enough information to do anything useful.
>>
>>> Besides, the reliable transport session does not invalidate the use of nonces and Map-Notifies as an indication that the MS has completely received the information, the just rely on the TCP state to know that nothing has changed.
>>
>> Right, so you have duplicate functionality. That isn’t efficient.
>>
>>>> I’d argue you may it worse. TCP does provide reliability but so does LISP itself. And the only reason the messages are periodic is because the spec said to send every 1 minute and timeout every 3 minutes. You can make it 1000 minutes and timeout every 3000 minutes.
>>>
>>> And this sounds as we would make the protocol very complicated to use (or code), as this would lead us to have to code/configure a specific registration pattern/logic for every use case that we want to support. Not saying we cannot, but it sounds like re-implementing TCP ;)
>>
>> You already have that code or you aren’t implementing Map-Register acks corrrectly. And changing a timer is really a constant change in the code. And what is spec’ed today in LISP proper is much simpler than TCP, it is not reimplementing it.
>>
>> But let's not drift from the point. The spec is written in a general way to solve only sending Map-Registers reliably. I suggest the text not mislead the reader to think this is a general new packet format for anything.
>>
>> When people judge which protocols they want to deploy, the people that do the due diligence will look at the protocol specs and see how much mechanism is designed in. And they make judgement decisions about using the protocol.
>>
>> I know of at least 2 vendors that said “I implemented pages 47-50 of the EVPN spec”.  ;-)
>>
>> In LISP. we want to make sure what we spec is what is used and not ignored or considered optional. So please document the specific use-case you want to implement. And I’d suggest making the draft name draft-*-lisp-reliable-registers.
>>
>>>> LISP can pack all those EID-records in a Map-Register just like TCP does. And if you want per nonce acks, you pack them in IP packets <= 65535 bytes. TCP will have to o that as well.
>>>
>>> This is exactly the point, while LISP signaling allows it we don’t need to re-implement every TCP feature in LISP, as TCP can already provide it.
>>
>> You don’t need to.
>>
>>>> And guess what. What if there is an RLOC-change and you already gave the last one to TCP and can’t pull it back. If you were waiting for an ack and a new RLOC-change came in (during a lossy case), you wouldn’t have to retransmit the old information wastily. So keep the “retransmission queue” in LISP has its advantages.
>>>
>>> I’m not sure this is so easy. UDP, just like TCP uses the RLOC (IP) as part of the “session identifier” and the nonce is per-packet, not per-session. The moment the RLOC changes on the xTR, the MS does not know that the xTR is the same so we’d need a retransmission process.
>>
>> I’m not talking about when the xTR changes, I’m talking about an address on an interface on the same xTR changes since it was DHCP’ed to the xTR or behind a NAT. But for the LISP-MN case, the xTR is moving and its RLOCs are changing. You couple this with pubsub and the extra Map-Registers with old RLOCs has a ripple effect where then the Map-Server sends Map-Notify messages to all the subscribers, then followed by another set of Map-Notifies with the new RLOC-set. That is a lot of (unnecessary) messaging and processing.
>>
>> We have to think about the implications of any one draft on the ENTIRE LISP architecture. It must work efficiently as one holistic distributed system.
>>
>> Dino
>>
>>>
>>> Marc
>>>
>>> On 12/5/17, 5:57 PM, "Dino Farinacci" <farinacci@gmail.com> wrote:
>>>
>>>> Dino,
>>>>
>>>> In addition to the previous arguments there are particular use-cases where the use of reliable transport simplified the deployment of LISP.
>>>
>>>    I understand its advantages. I am examining its costs.
>>>
>>>> As an example, the moment we started scaling datacenters to support 10s of thousands of hosts, the use of a reliable transport helped a lot the management of scale:
>>>> On one side it reduces the amount of signaling when nothing changes, since we use TCP state as an indication that xTRs and the MS are in sync and there is no need to deal with the optimization of the refresh logic (periodic or paced).
>>>
>>>    LISP (the application), does not know that itself, the xTR, is in sync with the map-server. The packets can be in flight or being retransmitted due to loss. But if a Map-Register is sent with a nonce and no Map-Notify is returned the xTR knows for sure the two are in sync.
>>>
>>>    I’d argue you may it worse. TCP does provide reliability but so does LISP itself. And the only reason the messages are periodic is because the spec said to send every 1 minute and timeout every 3 minutes. You can make it 1000 minutes and timeout every 3000 minutes.
>>>
>>>    So let’s keep periiodic overhead, reliability, and staying in sync as separate issues.
>>>
>>>> On the other side, with reliable transport we offload the reliable delivery of information (and congestion control)
>>>
>>>    I understand that. But you can’t say TCP is keeping you in sync, because you have removed detail from the applicationis.
>>>
>>>> from LISP to another process (TCP) that is entirely devoted and designed for this. For example, supporting events like mass VM moves relying purely on LISP based ACks became very challenging, as we ended up having to deal with congestion events related to the signaling load generated. The use of the reliable transport largely simplified the problem.
>>>
>>> Dino
>>>
>>>>
>>>> Marc
>>>>
>>>> On 12/5/17, 12:06 PM, "lisp on behalf of Johnson Leong (joleong)" <lisp-bounces@ietf.org on behalf of joleong@cisco.com> wrote:
>>>>
>>>>   Hi Dino,
>>>>
>>>>   A large portion of this draft discusses the state machine required for TCP and how to ensure the MS and xTR are in sync.  We literally reuse the entire UDP map-register code, we just wrap that message around the LISP TCP header so there's a lot of code reuse.  Finally, this draft is not meant to replace UDP register but in some of our use cases TCP would scale better to avoid the periodic registration.
>>>>
>>>>   -Johnson
>>>>
>>>>> On Dec 5, 2017, at 10:52 AM, Dino Farinacci <farinacci@gmail.com> wrote:
>>>>>
>>>>>> registration protocol, that might be orthogonal to other transport-related mechanisms. In my experience this has proved to be very effective in scalability of large LISP deployments, especially with the increased volume of registration data.
>>>>>
>>>>> I agree it’s a point solution for registration. Then why did you need to have a general format.
>>>>>
>>>>> I could support this draft if it was simplified to spec how to use Map-Registers in TCP and nothing more.
>>>>>
>>>>> The only thing I would add is how to use TLS so encryption is supported. More and more requirements are coming up for protecting the privacy of location information. And since Map-Registers carry RLOCs (and potential Geo-Coordnates) that information needs to be protected.
>>>>>
>>>>> Dino
>>>>> _______________________________________________
>>>>> lisp mailing list
>>>>> lisp@ietf.org
>>>>> https://www.ietf.org/mailman/listinfo/lisp
>>>>
>>>>   _______________________________________________
>>>>   lisp mailing list
>>>>   lisp@ietf.org
>>>>   https://www.ietf.org/mailman/listinfo/lisp
>>>>
>>>>
>>>
>>>
>>>
>>
> 
> _______________________________________________
> lisp mailing list
> lisp@ietf.org
> https://www.ietf.org/mailman/listinfo/lisp
>