Re: [storm] Send vs. Immediate

[Tom Talpey <[e-mail]>]

On 9/6/2015 5:58 AM, Elena Gurevich wrote:
> Hello dear authors,
>
> I am new in iWARP and need your help to clear my  misunderstanding of RFCs.
>
> RFC5041 Section 5.4 stands that:
>
> “At the Data Sink, DDP MUST Deliver a DDP Message if and only if all of
> the following are true:
>
> * the last DDP Segment of the DDP Message had its Last flag set,
>
> * all of the DDP Segments of the DDP Message have been Placed,
>
> * all preceding DDP Messages have been Placed, and
>
> * each preceding DDP Message has been Delivered to the ULP.”
>
> Let’s assume that data sink receives sequence of “rdma_write” and send”
> messages.
>
> According to RFC even if miss ordering happens “send” can be delivered
> to RDMAP only after last segment of rdma_write arrives and is placed.
>
> So RDMAP layer completes “send” only after full rdma_write assembly.

These statements are correct, the Send is delivered only after Placing
all the previous RDMA Write segments. This is a key behavior upon which
most upper layers depend.

>
> If so why we need to generate new “Immediate request type” and cannot
> reuse 8 bytes length “send” ?

I don't understand the question. An Immediate is another behavior,
and which is used by upper layers rather differently. Also, what
"8 bytes" are you referring to?

Perhaps you mean, why doesn't the upper layer use an RDMA Write with
Immediate? Some upper layers do, I believe Lustre does, for example,
but the original iWARP RDMAP (RFC5040) protocol did not support that
operation. The RFC7306 extension, published last year, adds it.

Other upper layers, for example, iSER, NFS/RDMA, SMB Direct, etc, need
a larger, separate message to complete an operation, and so use a full
Send. These protocols use iWARP, without Immediates.