Re: [rddp] DDP draft specification
David Robinson <David.Robinson@Sun.COM> Mon, 29 August 2005 04:03 UTC
Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1E9arb-0003YU-Hd; Mon, 29 Aug 2005 00:03:15 -0400
Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1E9arZ-0003YP-IO for rddp@megatron.ietf.org; Mon, 29 Aug 2005 00:03:13 -0400
Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA29655 for <rddp@ietf.org>; Mon, 29 Aug 2005 00:03:10 -0400 (EDT)
Received: from brmea-mail-4.sun.com ([192.18.98.36]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1E9aso-0000RN-1T for rddp@ietf.org; Mon, 29 Aug 2005 00:04:32 -0400
Received: from phys-aus08-1 ([129.153.131.88]) by brmea-mail-4.sun.com (8.12.10/8.12.9) with ESMTP id j7T437TY028114 for <rddp@ietf.org>; Sun, 28 Aug 2005 22:03:07 -0600 (MDT)
Received: from conversion-daemon.aus08-mail1.central.sun.com by aus08-mail1.central.sun.com (iPlanet Messaging Server 5.2 HotFix 1.24 (built Dec 19 2003)) id <0ILY00L01SPQCE@aus08-mail1.central.sun.com> (original mail from David.Robinson@Sun.COM) for rddp@ietf.org; Sun, 28 Aug 2005 23:03:06 -0500 (CDT)
Received: from [192.168.0.100] (vpn-129-153-214-72.Central.Sun.COM [129.153.214.72]) by aus08-mail1.central.sun.com (iPlanet Messaging Server 5.2 HotFix 1.24 (built Dec 19 2003)) with ESMTP id <0ILY00CUNTX6B7@aus08-mail1.central.sun.com>; Sun, 28 Aug 2005 23:03:06 -0500 (CDT)
Date: Sun, 28 Aug 2005 23:03:04 -0500
From: David Robinson <David.Robinson@Sun.COM>
Subject: Re: [rddp] DDP draft specification
In-reply-to: <0F31272A2BCBBE4FA01344C6E69DBF50295F4C@thoth.ivivity.com>
To: Sanjay Goyal <sanjayg@ivivity.com>
Message-id: <cb9b01186243c802751675ab64156bff@sun.com>
MIME-version: 1.0 (Apple Message framework v622)
X-Mailer: Apple Mail (2.622)
Content-type: text/plain; charset="US-ASCII"; format="flowed"
Content-transfer-encoding: 7bit
References: <0F31272A2BCBBE4FA01344C6E69DBF50295F4C@thoth.ivivity.com>
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 6ffdee8af20de249c24731d8414917d3
Content-Transfer-Encoding: 7bit
Cc: Caitlin Bestler <caitlinb@broadcom.com>, rddp@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: "IETF Remote Direct Data Placement \(rddp\) WG" <rddp.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>, <mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>, <mailto:rddp-request@ietf.org?subject=subscribe>
Sender: rddp-bounces@ietf.org
Errors-To: rddp-bounces@ietf.org
Sometime it helps to think of the architecture in its pure layered implementation instead of examine how it will actually be implemented to understand the distinction between placement and delivery. The primary requirement is that the LLP is reliable and an in order delivery protocol. In a pure implementation DDP will *never* see a packet that is not in order or out of sequence. In this pure case the distinction between delivery and placement is moot. But the impact of such pure layering is that an out of order segment will cause the LLP to force a retransmit or perform a copy to an intermediate buffer before delivery (i.e. SACK). This is a perfectly acceptable solution but suffers from either decreased bandwidth due to retransmissions or increased overhead due to copies and buffer consumption. Both of what DDP is trying to avoid. If you add the distinction of placement and delivery, a common implementation can mostly avoid the problems of out of order or lost segments. In a reliable LLP, before a delivery notification, an ULP can assume nothing about the contents of a buffer as they may change at any time up until delivery notification. The LLP can take advantage of this by working around the layering when it receives out of order data. It can examine the DDP headers to determine where the data should be placed to eliminate the intermediate buffering. But because it cannot make a delivery notification the ULPs can't know or reliably determine if placement has occurred. Once all of the intermediate segments have arrived then delivery notification can occur. There are things that can cause the placement to be invalidated, from the mundane bad CRC to something more complex like the connection reset before the missing segment arrives. Fortunately those things are very rare. > Or in other words MPA only delivers FPDUs to DDP and does not really > place the data. So the below sentence still does not make sense to me. > > This specification requires reliable, in order Delivery LLPs. > > Because the above sentence will mean that out-of-order DDP segment > will really never be received. So this does make sense if you consider that 'delivery' may not actual cause any data to be moved, it may simply be validation that a buffer is usable. So the LLP may 'place' data out of order or unreliably, if this speculative placement was correct then 'delivery' is a trivial operation. So your last statement is not correct, DDP will by definition *never* see an out of order DDP segment so the issue of if it was 'received' doesn't make sense. What it does see is an in order sequence of segments that if it violated the layering and snooped the memory buffers, it may detect that the memory was not filled in the expected ordering, or in some cases the same memory may have been written multiple times. There is no 'reception' until delivery, regardless of the contents of the destination buffers. -David > > > Best Regards, > Sanjay Goyal > > -----Original Message----- > From: Caitlin Bestler [mailto:caitlinb@broadcom.com] > Sent: Saturday, August 20, 2005 11:44 PM > To: Sanjay Goyal > Cc: rddp@ietf.org > Subject: RE: [rddp] DDP draft specification > > > > >> -----Original Message----- >> From: rddp-bounces@ietf.org [mailto:rddp-bounces@ietf.org] On >> Behalf Of Sanjay Goyal >> Sent: Friday, August 19, 2005 10:29 AM >> To: rddp@ietf.org >> Subject: [rddp] DDP draft specification >> >> Hi, >> >> The below 2 sentences are from draft-ietf-rddp-ddp-05.txt >> >> 1. DDP provides enough information in each DDP Segment to >> allow the ULP Payload in each inbound DDP Segment payloads to >> be directly Placed into the correct ULP Buffer, even when the >> DDP Segments arrive out-of-order. >> 2. This specification requires reliable, in order Delivery LLPs. >> >> First sentence says that DDP segments which are MPA FPDUs in >> TCP case can arrive out-of-order and second sentence says >> that LLP (MPA layer) needs to provide ordered delivery. Isn't >> it contradictory? >> >> Regards, >> >> Sanjay >> > > The first clause allows DDP to place data into buffers > in the order that the LLP provides DDP Segments for the > purpose of placement. > > The second clause specifies that the LLP will only > *deliver* the DDP Segments in order. DDP itself provides > for in-order delivery only after LLP delivery. > > Placement deals with buffering. Placement alone does > not enable the data to be used. See the rules about > buffers being undefined until delivery. > > Basically the same distinction between placement and > delivery that applies for DDP can apply between the > LLP and DDP. But that is implementation specific since > there is no standardization of the DDP/LLP interface. > It was a design goal that the classic sockets interface > be an acceptable interface, but it is no way mandated > or even suggested. > > _______________________________________________ > rddp mailing list > rddp@ietf.org > https://www1.ietf.org/mailman/listinfo/rddp _______________________________________________ rddp mailing list rddp@ietf.org https://www1.ietf.org/mailman/listinfo/rddp
- [rddp] DDP draft specification Sanjay Goyal
- Re: [rddp] DDP draft specification Sukanta ganguly
- Re: [rddp] DDP draft specification Caitlin Bestler
- RE: [rddp] DDP draft specification Caitlin Bestler
- Re: [rddp] DDP draft specification Michael Krause
- RE: [rddp] DDP draft specification Sanjay Goyal
- Re: [rddp] DDP draft specification David Robinson