Re: [storm] Error behavior (RE: iSER - another try)

<david.black@emc.com> Tue, 24 April 2012 14:31 UTC

Return-Path: <david.black@emc.com>
X-Original-To: storm@ietfa.amsl.com
Delivered-To: storm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 624DF21F8829 for <storm@ietfa.amsl.com>; Tue, 24 Apr 2012 07:31:53 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.427
X-Spam-Level:
X-Spam-Status: No, score=-110.427 tagged_above=-999 required=5 tests=[AWL=0.171, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CnGvhSbZMR9S for <storm@ietfa.amsl.com>; Tue, 24 Apr 2012 07:31:48 -0700 (PDT)
Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by ietfa.amsl.com (Postfix) with ESMTP id 38E4D21F881B for <storm@ietf.org>; Tue, 24 Apr 2012 07:31:44 -0700 (PDT)
Received: from hop04-l1d11-si01.isus.emc.com (HOP04-L1D11-SI01.isus.emc.com [10.254.111.54]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q3OEVhsB004263 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Apr 2012 10:31:44 -0400
Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.222.130]) by hop04-l1d11-si01.isus.emc.com (RSA Interceptor); Tue, 24 Apr 2012 10:31:24 -0400
Received: from mxhub19.corp.emc.com (mxhub19.corp.emc.com [10.254.93.48]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q3OEVNSg000800; Tue, 24 Apr 2012 10:31:23 -0400
Received: from mx15a.corp.emc.com ([169.254.1.107]) by mxhub19.corp.emc.com ([10.254.93.48]) with mapi; Tue, 24 Apr 2012 10:31:23 -0400
From: david.black@emc.com
To: ttalpey@microsoft.com
Date: Tue, 24 Apr 2012 10:31:21 -0400
Thread-Topic: Error behavior (RE: [storm] iSER - another try)
Thread-Index: AQHNIZtlCbqotxS8BECRMhFFRIFk1Zap/ttggAAJg3A=
Message-ID: <8D3D17ACE214DC429325B2B98F3AE71203502B38@MX15A.corp.emc.com>
References: <8D3D17ACE214DC429325B2B98F3AE7120925D1@MX15A.corp.emc.com> <CAP_=6dJT62LRDQSOTdOk1LvSxATcSDC4NjnDsXLE1fpOD21Gig@mail.gmail.com> <F83812DF4B59B9499C1BC978336D917461EA69D4@TK5EX14MBXC113.redmond.corp.microsoft.com> <8D3D17ACE214DC429325B2B98F3AE71203502ABC@MX15A.corp.emc.com> <F83812DF4B59B9499C1BC978336D917461EA74AC@TK5EX14MBXC113.redmond.corp.microsoft.com>
In-Reply-To: <F83812DF4B59B9499C1BC978336D917461EA74AC@TK5EX14MBXC113.redmond.corp.microsoft.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US
Content-Type: multipart/alternative; boundary="_000_8D3D17ACE214DC429325B2B98F3AE71203502B38MX15Acorpemccom_"
MIME-Version: 1.0
X-EMM-MHVC: 1
Cc: storm@ietf.org
Subject: Re: [storm] Error behavior (RE: iSER - another try)
X-BeenThere: storm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Storage Maintenance WG <storm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/storm>, <mailto:storm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/storm>
List-Post: <mailto:storm@ietf.org>
List-Help: <mailto:storm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/storm>, <mailto:storm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 24 Apr 2012 14:31:54 -0000

> Sure, but the error being proposed is generated by the receiver, and therefore if detected it will
> be a fatal error (closing the connection). That's a rather inconvenient way to negotiate the facility.

Unfortunately, that's all we have.  If iWARP RDMAP implementations support the invalidate versions of
Send, I'd prefer to confine the "SHOULD NOT use" recommendation for SendInv to IB (we'd still need a
MUST requirement on initiators to ensure appropriate STag invalidation).

> Also, my reading of the Infiniband spec section 9.2.3 is that the responder is free to ignore
> the Solicited Event bit, which means IB may not support the "that won't work" detection. In
> other words, IB allows the responder to translate SE to ordinary Send.

Now that's interesting ... RDMAP should return an error in that case.  Based on the IB behavior,
I'm inclined to agree with your "SHOULD NOT use" suggestion for the two solicited event versions
of send (SendSE and SendInvSE).

> I would agree with the statement if it read "(lack of receiver support MAY result in an error
> sent back to the sender)". But would it be useful?

Ah, I see the point now ... I think you're right that explaining the possible error behavior (MAY)
without requiring it is the better approach.

Thanks,
--David

From: Tom Talpey [mailto:ttalpey@microsoft.com]
Sent: Tuesday, April 24, 2012 10:03 AM
To: Black, David
Cc: storm@ietf.org
Subject: RE: Error behavior (RE: [storm] iSER - another try)

Sure, but the error being proposed is generated by the receiver, and therefore if detected it will be a fatal error (closing the connection). That's a rather inconvenient way to negotiate the facility.

Also, my reading of the Infiniband spec section 9.2.3 is that the responder is free to ignore the Solicited Event bit, which means IB may not support the "that won't work" detection. In other words, IB allows the responder to translate SE to ordinary Send.

I would agree with the statement if it read "(lack of receiver support MAY result in an error sent back to the sender)". But would it be useful?


From: david.black@emc.com [mailto:david.black@emc.com]
Sent: Monday, April 23, 2012 5:52 PM
To: Tom Talpey
Cc: storm@ietf.org
Subject: Error behavior (RE: [storm] iSER - another try)

> Also, isn't the following a new requirement on the RDMA layer?
> >(lack of receiver support MUST result in an error send back to the sender

For iWARP that should be an Unexpected Opcode error.  For other RCaPs, it should
be that or a "that won't work" error at the sender if the API is invocable.

Thanks,
--David

From: Tom Talpey [mailto:ttalpey@microsoft.com]<mailto:[mailto:ttalpey@microsoft.com]>
Sent: Monday, April 23, 2012 5:31 PM
To: Michael Ko; Black, David
Cc: storm@ietf.org<mailto:storm@ietf.org>
Subject: RE: [storm] iSER - another try

We need to separate the issues of Solicited Events from Invalidations.

I don't believe that the use of SendSE is somehow detectable with this approach. The conclusion from item ii) is that the iSER initiator SHOULD NOT arm for solicited events - it MUST be prepared for any event, whether solicited or not. This in turn implies that senders (targets) SHOULD NOT use SendSE at all - unless it's certain that *both* the RCaP and the initiator support them. This feature can neither be detected, nor is it negotiated by iSER.

The "invalidate" form is, however, sensible to allow after carefully documenting the requirement. The initiator MUST verify that the appropriate memory region was invalidated, if any. However, whether it MUST or SHOULD perform some additional action is going to be a tough choice. A MUST could be too strong of a requirement, if the implementation chooses to cache or share mappings. On the other hand, SHOULD can introduce security vulnerabilities. Thoughts?

Also,  isn't the following a new requirement on the RDMA layer?

>(lack of receiver support MUST result in an error send back to the sender).



From: storm-bounces@ietf.org<mailto:storm-bounces@ietf.org> [mailto:storm-bounces@ietf.org]<mailto:[mailto:storm-bounces@ietf.org]> On Behalf Of Michael Ko
Sent: Monday, April 23, 2012 11:55 AM
To: david.black@emc.com<mailto:david.black@emc.com>
Cc: storm@ietf.org<mailto:storm@ietf.org>
Subject: Re: [storm] iSER - another try

David,

I agree with this position.

Mike
On Mon, Apr 23, 2012 at 6:02 AM, <david.black@emc.com<mailto:david.black@emc.com>> wrote:
We don't seem to be making progress on definitive requirements that
vary by RCaP, so here's another suggestion (would be [4] if I were
numbering it ...)

It's clear that at least one implementation has diverged from the
requirements in RFC 5046 for use of SendSE, SendInv and SendInvSE.
I suggest documenting this and applying the IETF approach of "be
conservative in what you send, be liberal in what you accept",
roughly as follows:

i) Some iSER implementations have not followed RFC 5046's strict requirements
       for use of SendSE, SendInv and SendInvSE; they use Send instead.
ii) For interoperability, iSER implementations SHOULD accept and correctly
       process SendSE, SendInv and SendInvSE messages.
iii) SendSE, SendInv and SendInvSE should be regarded as optimizations or
       enhancements to the basic Send message, and their support varies by
       RCaP.  If these messages are used, the implementation SHOULD be
       capable of reverting to use of Send in order to work with a receiver
       that does not support these messages (lack of receiver support MUST
       result in an error send back to the sender).  These messages
       SHOULD NOT be used with the InfiniBand RCaP because InfiniBand does
       not require support for their functionality.
iv) New iSER implementations SHOULD use Send (and not these three additional
       messages) unless there are compelling reasons for doing otherwise
       (latter is implicit in use of "SHOULD", but worth saying explicitly,
       IMHO).
v) Some new text will be needed (including Security Considerations) to
       make it clear that STag invalidation is necessary for security
       reasons, and has to be performed independent of whether an
       Invalidate version of Send was used or not.

Is this workable?

Thanks,
--David
----------------------------------------------------
David L. Black, Distinguished Engineer
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953<tel:%2B1%20%28508%29%20293-7953>             FAX: +1 (508) 293-7786<tel:%2B1%20%28508%29%20293-7786>
david.black@emc.com<mailto:david.black@emc.com>        Mobile: +1 (978) 394-7754<tel:%2B1%20%28978%29%20394-7754>
----------------------------------------------------

_______________________________________________
storm mailing list
storm@ietf.org<mailto:storm@ietf.org>
https://www.ietf.org/mailman/listinfo/storm