Re: [rddp] DDP draft specification

David Robinson <David.Robinson@Sun.COM> Mon, 29 August 2005 04:03 UTC

Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1E9arb-0003YU-Hd; Mon, 29 Aug 2005 00:03:15 -0400
Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1E9arZ-0003YP-IO for rddp@megatron.ietf.org; Mon, 29 Aug 2005 00:03:13 -0400
Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA29655 for <rddp@ietf.org>; Mon, 29 Aug 2005 00:03:10 -0400 (EDT)
Received: from brmea-mail-4.sun.com ([192.18.98.36]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1E9aso-0000RN-1T for rddp@ietf.org; Mon, 29 Aug 2005 00:04:32 -0400
Received: from phys-aus08-1 ([129.153.131.88]) by brmea-mail-4.sun.com (8.12.10/8.12.9) with ESMTP id j7T437TY028114 for <rddp@ietf.org>; Sun, 28 Aug 2005 22:03:07 -0600 (MDT)
Received: from conversion-daemon.aus08-mail1.central.sun.com by aus08-mail1.central.sun.com (iPlanet Messaging Server 5.2 HotFix 1.24 (built Dec 19 2003)) id <0ILY00L01SPQCE@aus08-mail1.central.sun.com> (original mail from David.Robinson@Sun.COM) for rddp@ietf.org; Sun, 28 Aug 2005 23:03:06 -0500 (CDT)
Received: from [192.168.0.100] (vpn-129-153-214-72.Central.Sun.COM [129.153.214.72]) by aus08-mail1.central.sun.com (iPlanet Messaging Server 5.2 HotFix 1.24 (built Dec 19 2003)) with ESMTP id <0ILY00CUNTX6B7@aus08-mail1.central.sun.com>; Sun, 28 Aug 2005 23:03:06 -0500 (CDT)
Date: Sun, 28 Aug 2005 23:03:04 -0500
From: David Robinson <David.Robinson@Sun.COM>
Subject: Re: [rddp] DDP draft specification
In-reply-to: <0F31272A2BCBBE4FA01344C6E69DBF50295F4C@thoth.ivivity.com>
To: Sanjay Goyal <sanjayg@ivivity.com>
Message-id: <cb9b01186243c802751675ab64156bff@sun.com>
MIME-version: 1.0 (Apple Message framework v622)
X-Mailer: Apple Mail (2.622)
Content-type: text/plain; charset="US-ASCII"; format="flowed"
Content-transfer-encoding: 7bit
References: <0F31272A2BCBBE4FA01344C6E69DBF50295F4C@thoth.ivivity.com>
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 6ffdee8af20de249c24731d8414917d3
Content-Transfer-Encoding: 7bit
Cc: Caitlin Bestler <caitlinb@broadcom.com>, rddp@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: "IETF Remote Direct Data Placement \(rddp\) WG" <rddp.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>, <mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>, <mailto:rddp-request@ietf.org?subject=subscribe>
Sender: rddp-bounces@ietf.org
Errors-To: rddp-bounces@ietf.org

Sometime it helps to think of the architecture in its pure layered
implementation instead of examine how it will actually
be implemented to understand the distinction between placement
and delivery.

The primary requirement is that the LLP is reliable and
an in order delivery protocol.  In a pure implementation
DDP will *never* see a packet that is not in order or
out of sequence.  In this pure case the distinction between
delivery and placement is moot.  But the impact of such pure
layering is that an out of order segment will cause the LLP
to force a retransmit or perform a copy to an intermediate
buffer before delivery (i.e. SACK). This is a perfectly acceptable
solution but suffers from either decreased bandwidth due to
retransmissions or increased overhead due to copies and buffer
consumption.  Both of what DDP is trying to avoid.

If you add the distinction of placement and delivery, a common
implementation can mostly avoid the problems of out of
order or lost segments. In a reliable LLP, before a delivery
notification, an ULP can assume nothing about the contents
of a buffer as they may change at any time up until
delivery notification. The LLP can take advantage of this
by working around the layering when it receives out
of order data. It can examine the DDP headers to determine
where the data should be placed to eliminate the intermediate
buffering.  But because it cannot make a delivery notification
the ULPs can't know or reliably determine if placement has
occurred. Once all of the intermediate segments have arrived
then delivery notification can occur. There are things that
can cause the placement to be invalidated, from the mundane
bad CRC to something more complex like the connection reset
before the missing segment arrives. Fortunately those things
are very rare.

>  Or in other words MPA only delivers FPDUs to DDP and does not really 
> place the data. So the below sentence still does not make sense to me.
>
>  This specification requires reliable, in order Delivery LLPs.
>
> Because the above sentence will mean that out-of-order DDP segment 
> will really never be received.

So this does make sense if you consider that 'delivery' may not
actual cause any data to be moved, it may simply be validation that
a buffer is usable. So the LLP may 'place' data out of order or
unreliably, if this speculative placement was correct then
'delivery' is a trivial operation. So your last statement
is not correct, DDP will by definition *never* see an out of order DDP
segment so the issue of if it was 'received' doesn't make sense. What
it does see is an in order sequence of segments that if it violated
the layering and snooped the memory buffers, it may detect that the
memory was not filled in the expected ordering, or in some cases the
same memory may have been written multiple times.  There is no
'reception' until delivery, regardless of the contents of the
destination buffers.

	-David

>
>
> Best Regards,
> Sanjay Goyal
>
> -----Original Message-----
> From: Caitlin Bestler [mailto:caitlinb@broadcom.com]
> Sent: Saturday, August 20, 2005 11:44 PM
> To: Sanjay Goyal
> Cc: rddp@ietf.org
> Subject: RE: [rddp] DDP draft specification
>
>
>
>
>> -----Original Message-----
>> From: rddp-bounces@ietf.org [mailto:rddp-bounces@ietf.org] On
>> Behalf Of Sanjay Goyal
>> Sent: Friday, August 19, 2005 10:29 AM
>> To: rddp@ietf.org
>> Subject: [rddp] DDP draft specification
>>
>> Hi,
>>
>> The below 2 sentences are from draft-ietf-rddp-ddp-05.txt
>>
>> 1. DDP provides enough information in each DDP Segment to
>> allow the ULP Payload in each inbound DDP Segment payloads to
>> be directly Placed into the correct ULP Buffer, even when the
>> DDP Segments arrive out-of-order.
>> 2. This specification requires reliable, in order Delivery LLPs.
>>
>> First sentence says that DDP segments which are MPA FPDUs in
>> TCP case can arrive out-of-order and second sentence says
>> that LLP (MPA layer) needs to provide ordered delivery. Isn't
>> it contradictory?
>>
>> Regards,
>>
>> Sanjay
>>
>
> The first clause allows DDP to place data into buffers
> in the order that the LLP provides DDP Segments for the
> purpose of placement.
>
> The second clause specifies that the LLP will only
> *deliver* the DDP Segments in order. DDP itself provides
> for in-order delivery only after LLP delivery.
>
> Placement deals with buffering. Placement alone does
> not enable the data to be used. See the rules about
> buffers being undefined until delivery.
>
> Basically the same distinction between placement and
> delivery that applies for DDP can apply between the
> LLP and DDP. But that is implementation specific since
> there is no standardization of the DDP/LLP interface.
> It was a design goal that the classic sockets interface
> be an acceptable interface, but it is no way mandated
> or even suggested.
>
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp


_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp