[storm] more comments on http://tools.ietf.org/html/draft-ietf-storm-rdmap-ext-00

[Bernard Metzler <[e-mail]>]

Some comments on the RDMA Protocol Extension draft
(sorry when partially overlapping with Arkady's email
from 3 May).



The terms "Data Source" and "Data Sink" are used for Immediate
Data and Atomic Operations as well. For Atomics I found it
confusing and would prefer to have the 'Requester' and 'Responder'
terms used throughout the whole document.


Immediate Data
The specification introduces a new RDMAP message format.
Section 1 and 6 are limiting its usage to Writes, eg.:
"Immediate Data messages allow the ULP at the sender to
provide a small amount of data following an RDMA Write payload."
Why not Sends, while SendWithImm is defined for other RDMA
transports and is part of a well established RDMA verbs
interface (OpenFabrics)?


While potentially out of the primary business of the
specification of a wire protocol - I assume Immediate Data
will consume a peers Receive Queue Element?
A short discussion including appropriate RQ sizing might
be helpful for the reader (and implementer). It might go
into section 6.1.



Atomics
5.1.1: I found it difficult to understand the FetchAdd decription.
In particular, the statement of
"The setting of "Add Mask" field to 0x0000000000000000 results
in Atomic Add of 64-bit Original Remote Data Value and 64-bit
"Add Data"."
was surprising to me. I would have expected all bits set to 1.

FetchAdd sometimes extends to FetchAndAdd. I would prefer one term
only.

The abbreviation of RMW seem to refer to ReadModifyWrite.
Better to be written in full length? It could be confused
with RemoteMemoryWindow.



Ordering and Completion Table
Section 7 provides a table of operation ordering.
For Immediate Data, I found the term "Placement" inappropriate.
Immediate Data are not placed in a target buffer. Immediate Data
are received and delivered to the ULP. Maybe exchanging "Immediate
Data is Placed at Remote Peer" with "Immediate Data is received
at Remote Peer" would clarify? I see the same issue for Atomic
Responses, which are also not Placed but received.

The discussion of the two consecutive Atomic operations (page 22)
seem to allow re-ordering of Atomic responses. Is that true?
It might complicate response processing. 



General/Implementation
It might be a good idea to try an implementation of
the proposed RDMAP extension. The SoftiWARP stack might
be a good candidate for that.


Thanks,
Bernard.