Re: [storm] WG Last Call - iSER - comments

<david.black@emc.com> Fri, 09 December 2011 22:53 UTC

Return-Path: <david.black@emc.com>
X-Original-To: storm@ietfa.amsl.com
Delivered-To: storm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8326C21F84D4 for <storm@ietfa.amsl.com>; Fri, 9 Dec 2011 14:53:11 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -106.447
X-Spam-Level:
X-Spam-Status: No, score=-106.447 tagged_above=-999 required=5 tests=[AWL=0.151, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vpgavN-pUK5P for <storm@ietfa.amsl.com>; Fri, 9 Dec 2011 14:53:05 -0800 (PST)
Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id B5AFE21F84D2 for <storm@ietf.org>; Fri, 9 Dec 2011 14:53:01 -0800 (PST)
Received: from hop04-l1d11-si04.isus.emc.com (HOP04-L1D11-SI04.isus.emc.com [10.254.111.24]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id pB9Mqu3W001117 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 9 Dec 2011 17:52:56 -0500
Received: from mailhub.lss.emc.com (mailhubhoprd03.lss.emc.com [10.254.221.145]) by hop04-l1d11-si04.isus.emc.com (RSA Interceptor); Fri, 9 Dec 2011 17:52:39 -0500
Received: from mxhub05.corp.emc.com (mxhub05.corp.emc.com [128.222.70.202]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id pB9MqcYh029502; Fri, 9 Dec 2011 17:52:38 -0500
Received: from mx14a.corp.emc.com ([169.254.1.79]) by mxhub05.corp.emc.com ([128.222.70.202]) with mapi; Fri, 9 Dec 2011 17:52:38 -0500
From: david.black@emc.com
To: Michael@huaweisymantec.com, storm@ietf.org
Date: Fri, 09 Dec 2011 17:52:38 -0500
Thread-Topic: [storm] WG Last Call - iSER - comments
Thread-Index: Acy2EYdKpg2lTxofR66NX1aiM+JWnQAqD7pQ
Message-ID: <7C4DFCE962635144B8FAE8CA11D0BF1E05A000CCD7@MX14A.corp.emc.com>
References: <7C4DFCE962635144B8FAE8CA11D0BF1E059E2706D3@MX14A.corp.emc.com> <7C4DFCE962635144B8FAE8CA11D0BF1E059FEA5D20@MX14A.corp.emc.com> <0B5A79320CF9409689D63C152AC9418C@china.huawei.com> <7C4DFCE962635144B8FAE8CA11D0BF1E059FEA612D@MX14A.corp.emc.com> <C460CF1609BC426185ADD8DF995A5A86@china.huawei.com>
In-Reply-To: <C460CF1609BC426185ADD8DF995A5A86@china.huawei.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US
Content-Type: multipart/alternative; boundary="_000_7C4DFCE962635144B8FAE8CA11D0BF1E05A000CCD7MX14Acorpemcc_"
MIME-Version: 1.0
X-EMM-MHVC: 1
Subject: Re: [storm] WG Last Call - iSER - comments
X-BeenThere: storm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Storage Maintenance WG <storm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/storm>, <mailto:storm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/storm>
List-Post: <mailto:storm@ietf.org>
List-Help: <mailto:storm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/storm>, <mailto:storm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 09 Dec 2011 22:53:11 -0000

Mike,

[still with WG chair hat off]

Your proposed text changes for items 6 and 7 are fine with me.

For item 5, I think it's important that Section 2.3 item 4 make it clear that for iSER, the SCSI Data-out PDU is *not* a data-type PDU.  My proposed text was a bit weak on that point, but I didn't see it at all in your proposed text.

Here's another attempt at new text (combining text from both of us and editing further):

   4. RDMA Write, RDMA Read Request, and RDMA Read Response Messages
      are used for carrying control and all data information associated
      with the iSCSI data-type PDUs (i.e., SCSI Data-In PDUs and R2T PDUs).
As described in section 7.3.4, R2Ts are transformed by the iSER layer
at the target into RDMA Read operations. Therefore, iSER sends
solicited SCSI write data using RDMA Read Response Messages instead
of SCSI Data-out PDUs.  iSER uses SCSI Data-out PDUs only for unsolicited
SCSI Write data, which is sent via Send Messages instead of RDMA.  Because
RDMA is not used, the SCSI Data-out PDUs for unsolicited data are not
considered to be iSCSI data-type PDUs for iSER. See section 7.1 for more
discussion of iSCSI data-type PDUs.

Unfortunately, this is still somewhat subtle and potentially confusing (the Data-out PDU is not a data-type PDU, rather the Data-out PDU is a control-type PDU ... huh??); the alternative is probably a serious rewrite to replace the terms "data-type PDU" and "control-type PDU" with terms that don't use "data" and "control", e.g., "RDMA-transfer PDU" and "message-transfer PDU".

What do you think?

Thanks,
--David

From: Michael Ko [mailto:Michael@huaweisymantec.com]
Sent: Thursday, December 08, 2011 8:22 PM
To: Black, David; storm@ietf.org
Subject: Re: [storm] WG Last Call - iSER - comments

David,

My comments are inline.  If this is acceptable to you, I can upload the revised draft.

Mike
----- Original Message -----
From: david.black@emc.com<mailto:david.black@emc.com>
To: Michael@huaweisymantec.com<mailto:Michael@huaweisymantec.com> ; storm@ietf.org<mailto:storm@ietf.org>
Sent: Monday, December 05, 2011 11:26 PM
Subject: RE: [storm] WG Last Call - iSER - comments

Hi Mike,

Thanks for the quick response - your changes for items 1-4 and 8 in the -06 version look fine to me.

For item 5, I see where I got confused - the definitions section (1.1) and Section 7.1 both define the SCSI Data-out PDU to not be a data-type PDU.  That's somewhat counter-intuitive, but can be handled with some clarifying text.  I suggest the following clarifying change to item 4 in section 2.3 (my new text probably isn't 100% correct, but should be close):

OLD
   4. RDMA Write, RDMA Read Request, and RDMA Read Response Messages
      are used for carrying control and all data information associated
      with the iSCSI data-type PDUs.  See section 7.1 for more details
      on iSCSI data-type PDUs.
NEW
   4. RDMA Write, RDMA Read Request, and RDMA Read Response Messages
      are used for carrying control and all data information associated
      with the iSCSI data-type PDUs (i.e., SCSI Data-In PDUs and R2T PDUs).
iSCSI SCSI Data-out PDUs do not use RDMA transfer mechanisms because
the RDMA Read operation that replaces the R2T PDU obviates the use
of SCSI Data-out PDUs for solicited Data, and SCSI Data-out
PDUs for unsolicited data are transferred by Send Messages instead of
using RDMA.  See section 7.1 for more details on iSCSI data-type PDUs.
END

[mk] I have added the following at the end of item 4: "As described in section 7.3.4, R2Ts are transformed by the iSER layer at the target into RDMA Read operations.  Therefore solicited SCSI write data are sent using RDMA Read Response Messages instead of the SCSI Data-out PDUs.  For unsolicited SCSI Write data, the iSER Layer at the initiator uses Send Messages to send the SCSI Data-out PDUs."

For item 6 - if the connection starts in RDMA mode, how is login negotiation conducted?  I think the answer is Send Messages.  In any case, a sentence or two that answers this question in the first paragraph of Section 5.1 is what I'm looking for, as it's easy to mis-read that paragraph as requiring a start in normal TCP mode, not RDMA mode.

[mk] I have added the following paragraph after the first paragraph in section 5: "For a transport layer that operates in byte stream mode such as TCP, the RCaP implementation supports the enabling of the RDMA mode after Connection establishment and the exchange of Login parameters in byte stream mode.  For a transport layer that provides message delivery capability such as [IB], the RCaP implementation supports the use of the messaging capability by the iSCSI Layer directly for the Login phase after connection establishment before enabling iSER-assisted mode."

For item 7, the new text says:

      If iSERHelloRequired is negotiated to "No", then the maximum number
      of RDMA Read operations that can be active is negotiated via other
      means.

I suggest changing "other means" to "other means outside the scope of this document" and providing an example of "other means".

[mk] I have added the following after "other means": "outside the scope of this document.  For example, in InfiniBand, iSER connection setup uses InfiniBand CM MADs, with additional iSER information exchanged in the private data."

Thanks,
--David

From: Michael Ko [mailto:Michael@huaweisymantec.com]
Sent: Sunday, December 04, 2011 3:32 PM
To: Black, David; storm@ietf.org
Subject: Re: [storm] WG Last Call - iSER - comments

David,

Thanks for the review.  My comments  are inline.

Mike
----- Original Message -----
From: david.black@emc.com<mailto:david.black@emc.com>
To: storm@ietf.org<mailto:storm@ietf.org>
Sent: Friday, December 02, 2011 4:00 PM
Subject: [storm] WG Last Call - iSER - comments

This email is sent with WG chair hat off - all of these comments are for further discussion,
but they do need to be dealt with.

In addition to my previous comment on references:

> The iSER draft currently references RFC 3720 for iSCSI - that reference will need to be
> updated to at least the new consolidated iSCSI draft, and a reference to the iSCSI new
> features (SAM) draft should probably be added.

[mk] Done.

I've now done a relatively thorough review of draft-ietf-storm-iser-05.  It looks very good
overall, but I did find a number of additional things that need attention (items 5,6 and 8
are tagged as Major items):

1) Nit - In section 1.1's definition of connection, change "logical circuit" to "logical
bi-directional communication channel".

[mk] Done

2) The SAM-2 reference needs to be updated to the version of SAM-5 that the iSCSI SAM
new features draft uses, and that new features draft does need to be added as a normative
reference.

[mk] Done

3) There's a discussion of markers in Section 2.1.  I'd prefer that it be removed, but I
could live with it remaining, although it would have to informatively reference RFC 3720,
not the new consolidated iSCSI draft.

[mk] I have removed all references to markers.

4) In section 2.3, items 1 and 2 are inconsistent, as they use "session" and "connection"
for essentially the same concept.  I suggest changing "session" to "connection in item 1.

[mk] iSER-assisted mode is negotiated using the RDMAExtensions key in the leading connection, as stated in sections 5.1 and 6.3, to ensure that "an entire iSCSI session can only operate in one mode".  Hence the choice of the words "session" and "connection" in items 1 and 2 are correct.  I have added "in the leading connection" before "for each session" in the first sentence in item 1.

5) Major: There's something wrong with the discussion of how to send unsolicited data.
Item 4 in section 2.3 requires use of tagged buffers (RDMA), but the second paragraph in
Section 2.6 requires use of Send (untagged buffers, not RDMA), the new key in 6.9 appears
to allow unsolicited data in a tagged buffer (RDMA), but the next to last paragraph in
7.3.4 requires use of Send (untagged buffers, not RDMA).

[mk] Item 4 states that the control and all data information associated with the iSCSI data-type PDUs are handled in iSER using RDMA Write, RDMA Read Request and RDMA Read Response Messages.  The reader is referred to section 7.1 for the meaning of "iSCSI data-type PDUs", but item 4 itself does not mention tagged buffers.  I have reworded item 4 for clarity.  The new key in 6.9 is intended to resolve the issue of how the I/O buffer is used.  In iSCSI (RFC3270), the I/O buffer is used to contain all the data associated with the SCSI operation.  Some of this data can be transferred in an unsolicited fashion (using Send Messages), while the rest is transferred using RDMA.  This new key restricts of the use of this buffer for solicited data only, as stated in 6.9.  2.6 and 7.3.4 are correct as stated.

6) Major: I think something is missing in Section 5 to explains how to conduct iSCSI
negotiation when the connections start up in iSER-assisted mode.  I assume that this is
done via Send messages and the resource allocation referred to is the resources for RDMA,
but that does need to be explained.

[mk] In this version, iSER is changed to remove the requirement that the connection transitions from TCP mode to RDMA mode.  It does not require that login negotiations be done using Send Messages.

7) The first paragraph of Section 8.2 describes what happens when iSERHelloRequired is
negotiated to "Yes" - add a few sentence to explain what happens when it's negotiated
to "No", which is the typical case for implementations.  A similar problem occurs in
10.1.3.4 - that one's easily handled by saying that these errors cannot occur in the "No"
case.  Please check for all other dependencies on the negotiated value of
iSERHelloRequired, and make sure that both the "Yes" and "No" cases are covered.

[mk] Done

8) Major: The IANA Considerations section is empty.  That is wrong - the new keys defined
in sections 6.8 - 6.10 need to be registered in the iSCSI Login/Text Keys registry at: http://www.iana.org/assignments/iscsi-parameters
IANA also should be requested to update the registrations of the other 4 iSER keys in that
registry to reference the RFC number of this draft when it is published as an RFC.

[mk] Done.

Thanks,
--David
----------------------------------------------------
David L. Black, Distinguished Engineer
EMC Corporation, 176 South St., Hopkinton, MA 01748
+1 (508) 293-7953 FAX: +1 (508) 293-7786
david.black@emc.com<mailto:david.black@emc.com> Mobile: +1 (978) 394-7754
----------------------------------------------------

_______________________________________________
storm mailing list
storm@ietf.org<mailto:storm@ietf.org>
https://www.ietf.org/mailman/listinfo/storm