Re: [storm] WG Last Call - iSER - comments

[Michael Ko <[e-mail]>]

David,

I think the key here is to go back to the definition of iSCSI data-type PDU 
(section 1.1), which states that an iSCSI data-type PDU is not delivered 
as-is to the remote iSCSI Layer since the associated data transfer is done 
without the participation of the remote iSCSI Layer.  So rewriting the text 
with the rationale of the definition in mind, here is a revised take:

RDMA Write, RDMA Read Request, and RDMA Read Response Messages are used for 
carrying control and all data information associated with the iSCSI 
data-type PDUs (i.e., SCSI Data-In PDUs and R2T PDUs).  By definition (see 
section 1.1), an iSCSI data-type PDU is an iSCSI PDU that causes data 
transfer which is transparent to the remote iSCSI Layer.  As a result, the 
PDU itself is not delivered as-is to the remote iSCSI Layer.  In the case of 
R2Ts, the iSER layer at the target transforms the R2Ts into RDMA Read 
operations.  The solicited SCSI write data is sent by the iSER layer at the 
initiator using RDMA Read Response Messages instead of SCSI Data-out PDUs. 
This is not the case for unsolicited SCSI Write data, which is sent by the 
iSCSI layer as a SCSI Data-out PDU and is sent as-is by the iSER layer using 
a Send Message.  Because the iSCSI layer is invovled in the handling of the 
unsolicited SCSI write data, the SCSI Data-out PDU for unsolicited data is 
not considered an iSCSI data-type PDU.  See section 7.1 for more discussion 
of iSCSI data-type PDUs.

What do you think?

Mike

----- Original Message ----- 
From: [e-mail]
To: [e-mail] ; [e-mail]
Sent: Friday, December 09, 2011 2:52 PM
Subject: RE: [storm] WG Last Call - iSER - comments


Mike,



[still with WG chair hat off]



Your proposed text changes for items 6 and 7 are fine with me.



For item 5, I think it's important that Section 2.3 item 4 make it clear 
that for iSER, the SCSI Data-out PDU is *not* a data-type PDU.  My proposed 
text was a bit weak on that point, but I didn't see it at all in your 
proposed text.



Here's another attempt at new text (combining text from both of us and 
editing further):



   4. RDMA Write, RDMA Read Request, and RDMA Read Response Messages

      are used for carrying control and all data information associated

      with the iSCSI data-type PDUs (i.e., SCSI Data-In PDUs and R2T PDUs).

As described in section 7.3.4, R2Ts are transformed by the iSER layer

at the target into RDMA Read operations. Therefore, iSER sends

solicited SCSI write data using RDMA Read Response Messages instead

of SCSI Data-out PDUs.  iSER uses SCSI Data-out PDUs only for unsolicited

SCSI Write data, which is sent via Send Messages instead of RDMA.  Because

RDMA is not used, the SCSI Data-out PDUs for unsolicited data are not

considered to be iSCSI data-type PDUs for iSER. See section 7.1 for more

discussion of iSCSI data-type PDUs.



Unfortunately, this is still somewhat subtle and potentially confusing (the 
Data-out PDU is not a data-type PDU, rather the Data-out PDU is a 
control-type PDU ... huh??); the alternative is probably a serious rewrite 
to replace the terms "data-type PDU" and "control-type PDU" with terms that 
don't use "data" and "control", e.g., "RDMA-transfer PDU" and 
"message-transfer PDU".



What do you think?



Thanks,
--David



From: Michael Ko [mailto:[e-mail]]
Sent: Thursday, December 08, 2011 8:22 PM
To: Black, David; [e-mail]
Subject: Re: [storm] WG Last Call - iSER - comments



David,



My comments are inline.  If this is acceptable to you, I can upload the 
revised draft.



Mike

----- Original Message ----- 

From: [e-mail]

To: [e-mail] ; [e-mail]

Sent: Monday, December 05, 2011 11:26 PM

Subject: RE: [storm] WG Last Call - iSER - comments



Hi Mike,



Thanks for the quick response - your changes for items 1-4 and 8 in the -06 
version look fine to me.



For item 5, I see where I got confused - the definitions section (1.1) and 
Section 7.1 both define the SCSI Data-out PDU to not be a data-type PDU. 
That's somewhat counter-intuitive, but can be handled with some clarifying 
text.  I suggest the following clarifying change to item 4 in section 2.3 
(my new text probably isn't 100% correct, but should be close):



OLD

   4. RDMA Write, RDMA Read Request, and RDMA Read Response Messages

      are used for carrying control and all data information associated

      with the iSCSI data-type PDUs.  See section 7.1 for more details

      on iSCSI data-type PDUs.

NEW

   4. RDMA Write, RDMA Read Request, and RDMA Read Response Messages

      are used for carrying control and all data information associated

      with the iSCSI data-type PDUs (i.e., SCSI Data-In PDUs and R2T PDUs).

iSCSI SCSI Data-out PDUs do not use RDMA transfer mechanisms because

the RDMA Read operation that replaces the R2T PDU obviates the use

of SCSI Data-out PDUs for solicited Data, and SCSI Data-out

PDUs for unsolicited data are transferred by Send Messages instead of

using RDMA.  See section 7.1 for more details on iSCSI data-type PDUs.

END



[mk] I have added the following at the end of item 4: "As described in 
section 7.3.4, R2Ts are transformed by the iSER layer at the target into 
RDMA Read operations.  Therefore solicited SCSI write data are sent using 
RDMA Read Response Messages instead of the SCSI Data-out PDUs.  For 
unsolicited SCSI Write data, the iSER Layer at the initiator uses Send 
Messages to send the SCSI Data-out PDUs."



For item 6 - if the connection starts in RDMA mode, how is login negotiation 
conducted?  I think the answer is Send Messages.  In any case, a sentence or 
two that answers this question in the first paragraph of Section 5.1 is what 
I'm looking for, as it's easy to mis-read that paragraph as requiring a 
start in normal TCP mode, not RDMA mode.



[mk] I have added the following paragraph after the first paragraph in 
section 5: "For a transport layer that operates in byte stream mode such as 
TCP, the RCaP implementation supports the enabling of the RDMA mode after 
Connection establishment and the exchange of Login parameters in byte stream 
mode.  For a transport layer that provides message delivery capability such 
as [IB], the RCaP implementation supports the use of the messaging 
capability by the iSCSI Layer directly for the Login phase after connection 
establishment before enabling iSER-assisted mode."



For item 7, the new text says:



      If iSERHelloRequired is negotiated to "No", then the maximum number

      of RDMA Read operations that can be active is negotiated via other

      means.



I suggest changing "other means" to "other means outside the scope of this 
document" and providing an example of "other means".



[mk] I have added the following after "other means": "outside the scope of 
this document.  For example, in InfiniBand, iSER connection setup uses 
InfiniBand CM MADs, with additional iSER information exchanged in the 
private data."



Thanks,
--David



From: Michael Ko [mailto:[e-mail]]
Sent: Sunday, December 04, 2011 3:32 PM
To: Black, David; [e-mail]
Subject: Re: [storm] WG Last Call - iSER - comments



David,



Thanks for the review.  My comments  are inline.



Mike

----- Original Message ----- 

From: [e-mail]

To: [e-mail]

Sent: Friday, December 02, 2011 4:00 PM

Subject: [storm] WG Last Call - iSER - comments



This email is sent with WG chair hat off - all of these comments are for 
further discussion,
but they do need to be dealt with.

In addition to my previous comment on references:

> The iSER draft currently references RFC 3720 for iSCSI - that reference 
> will need to be
> updated to at least the new consolidated iSCSI draft, and a reference to 
> the iSCSI new
> features (SAM) draft should probably be added.



[mk] Done.

I've now done a relatively thorough review of draft-ietf-storm-iser-05.  It 
looks very good
overall, but I did find a number of additional things that need attention 
(items 5,6 and 8
are tagged as Major items):

1) Nit - In section 1.1's definition of connection, change "logical circuit" 
to "logical
bi-directional communication channel".



[mk] Done

2) The SAM-2 reference needs to be updated to the version of SAM-5 that the 
iSCSI SAM
new features draft uses, and that new features draft does need to be added 
as a normative
reference.



[mk] Done

3) There's a discussion of markers in Section 2.1.  I'd prefer that it be 
removed, but I
could live with it remaining, although it would have to informatively 
reference RFC 3720,
not the new consolidated iSCSI draft.



[mk] I have removed all references to markers.

4) In section 2.3, items 1 and 2 are inconsistent, as they use "session" and 
"connection"
for essentially the same concept.  I suggest changing "session" to 
"connection in item 1.



[mk] iSER-assisted mode is negotiated using the RDMAExtensions key in the 
leading connection, as stated in sections 5.1 and 6.3, to ensure that "an 
entire iSCSI session can only operate in one mode".  Hence the choice of the 
words "session" and "connection" in items 1 and 2 are correct.  I have added 
"in the leading connection" before "for each session" in the first sentence 
in item 1.


5) Major: There's something wrong with the discussion of how to send 
unsolicited data.
Item 4 in section 2.3 requires use of tagged buffers (RDMA), but the second 
paragraph in
Section 2.6 requires use of Send (untagged buffers, not RDMA), the new key 
in 6.9 appears
to allow unsolicited data in a tagged buffer (RDMA), but the next to last 
paragraph in
7.3.4 requires use of Send (untagged buffers, not RDMA).



[mk] Item 4 states that the control and all data information associated with 
the iSCSI data-type PDUs are handled in iSER using RDMA Write, RDMA Read 
Request and RDMA Read Response Messages.  The reader is referred to section 
7.1 for the meaning of "iSCSI data-type PDUs", but item 4 itself does not 
mention tagged buffers.  I have reworded item 4 for clarity.  The new key in 
6.9 is intended to resolve the issue of how the I/O buffer is used.  In 
iSCSI (RFC3270), the I/O buffer is used to contain all the data associated 
with the SCSI operation.  Some of this data can be transferred in an 
unsolicited fashion (using Send Messages), while the rest is transferred 
using RDMA.  This new key restricts of the use of this buffer for solicited 
data only, as stated in 6.9.  2.6 and 7.3.4 are correct as stated.

6) Major: I think something is missing in Section 5 to explains how to 
conduct iSCSI
negotiation when the connections start up in iSER-assisted mode.  I assume 
that this is
done via Send messages and the resource allocation referred to is the 
resources for RDMA,
but that does need to be explained.



[mk] In this version, iSER is changed to remove the requirement that the 
connection transitions from TCP mode to RDMA mode.  It does not require that 
login negotiations be done using Send Messages.

7) The first paragraph of Section 8.2 describes what happens when 
iSERHelloRequired is
negotiated to "Yes" - add a few sentence to explain what happens when it's 
negotiated
to "No", which is the typical case for implementations.  A similar problem 
occurs in
10.1.3.4 - that one's easily handled by saying that these errors cannot 
occur in the "No"
case.  Please check for all other dependencies on the negotiated value of
iSERHelloRequired, and make sure that both the "Yes" and "No" cases are 
covered.



[mk] Done

8) Major: The IANA Considerations section is empty.  That is wrong - the new 
keys defined
in sections 6.8 - 6.10 need to be registered in the iSCSI Login/Text Keys 
registry at: http://www.iana.org/assignments/iscsi-parameters
IANA also should be requested to update the registrations of the other 4 
iSER keys in that
registry to reference the RFC number of this draft when it is published as 
an RFC.



[mk] Done.

Thanks,
--David
----------------------------------------------------
David L. Black, Distinguished Engineer
EMC Corporation, 176 South St., Hopkinton, MA 01748
[phone] FAX: [phone]
[e-mail] Mobile: [phone]
----------------------------------------------------

_______________________________________________
storm mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/storm