Re: [storm] Invalidation (was RE: iSER - another try)

Tom Talpey <ttalpey@microsoft.com> Tue, 24 April 2012 14:09 UTC

Return-Path: <ttalpey@microsoft.com>
X-Original-To: storm@ietfa.amsl.com
Delivered-To: storm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 938F421F87E5 for <storm@ietfa.amsl.com>; Tue, 24 Apr 2012 07:09:14 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -103.598
X-Spam-Level:
X-Spam-Status: No, score=-103.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rJ2C8csQjx7h for <storm@ietfa.amsl.com>; Tue, 24 Apr 2012 07:09:11 -0700 (PDT)
Received: from va3outboundpool.messaging.microsoft.com (va3ehsobe005.messaging.microsoft.com [216.32.180.31]) by ietfa.amsl.com (Postfix) with ESMTP id 9E78521F87E1 for <storm@ietf.org>; Tue, 24 Apr 2012 07:09:10 -0700 (PDT)
Received: from mail35-va3-R.bigfish.com (10.7.14.235) by VA3EHSOBE003.bigfish.com (10.7.40.23) with Microsoft SMTP Server id 14.1.225.23; Tue, 24 Apr 2012 14:09:09 +0000
Received: from mail35-va3 (localhost [127.0.0.1]) by mail35-va3-R.bigfish.com (Postfix) with ESMTP id 32C3A2203A4 for <storm@ietf.org>; Tue, 24 Apr 2012 14:09:09 +0000 (UTC)
X-SpamScore: -36
X-BigFish: VS-36(zz9371Ic85fh1432N103eM98dK4015Izz1202hzz1033IL8275bh8275dhz2fh2a8h668h839hd25h)
X-Forefront-Antispam-Report: CIP:131.107.125.8; KIP:(null); UIP:(null); IPV:NLI; H:TK5EX14HUBC103.redmond.corp.microsoft.com; RD:none; EFVD:NLI
Received-SPF: pass (mail35-va3: domain of microsoft.com designates 131.107.125.8 as permitted sender) client-ip=131.107.125.8; envelope-from=ttalpey@microsoft.com; helo=TK5EX14HUBC103.redmond.corp.microsoft.com ; icrosoft.com ;
Received: from mail35-va3 (localhost.localdomain [127.0.0.1]) by mail35-va3 (MessageSwitch) id 1335276546681374_3266; Tue, 24 Apr 2012 14:09:06 +0000 (UTC)
Received: from VA3EHSMHS003.bigfish.com (unknown [10.7.14.238]) by mail35-va3.bigfish.com (Postfix) with ESMTP id 8DC8E4A004E; Tue, 24 Apr 2012 14:09:06 +0000 (UTC)
Received: from TK5EX14HUBC103.redmond.corp.microsoft.com (131.107.125.8) by VA3EHSMHS003.bigfish.com (10.7.99.13) with Microsoft SMTP Server (TLS) id 14.1.225.23; Tue, 24 Apr 2012 14:09:03 +0000
Received: from TK5EX14MBXC113.redmond.corp.microsoft.com ([169.254.6.69]) by TK5EX14HUBC103.redmond.corp.microsoft.com ([157.54.86.9]) with mapi id 14.02.0298.005; Tue, 24 Apr 2012 14:08:32 +0000
From: Tom Talpey <ttalpey@microsoft.com>
To: "david.black@emc.com" <david.black@emc.com>
Thread-Topic: Invalidation (was RE: [storm] iSER - another try)
Thread-Index: AQHNIZxawWEnUE5nAk2VVIJghKiXbpaqAr7A
Date: Tue, 24 Apr 2012 14:08:32 +0000
Message-ID: <F83812DF4B59B9499C1BC978336D917461EA74D7@TK5EX14MBXC113.redmond.corp.microsoft.com>
References: <8D3D17ACE214DC429325B2B98F3AE7120925D1@MX15A.corp.emc.com> <CAP_=6dJT62LRDQSOTdOk1LvSxATcSDC4NjnDsXLE1fpOD21Gig@mail.gmail.com> <F83812DF4B59B9499C1BC978336D917461EA69D4@TK5EX14MBXC113.redmond.corp.microsoft.com> <8D3D17ACE214DC429325B2B98F3AE71203502ABD@MX15A.corp.emc.com>
In-Reply-To: <8D3D17ACE214DC429325B2B98F3AE71203502ABD@MX15A.corp.emc.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [157.54.51.34]
Content-Type: multipart/alternative; boundary="_000_F83812DF4B59B9499C1BC978336D917461EA74D7TK5EX14MBXC113r_"
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
Cc: "storm@ietf.org" <storm@ietf.org>
Subject: Re: [storm] Invalidation (was RE: iSER - another try)
X-BeenThere: storm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Storage Maintenance WG <storm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/storm>, <mailto:storm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/storm>
List-Post: <mailto:storm@ietf.org>
List-Help: <mailto:storm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/storm>, <mailto:storm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 24 Apr 2012 14:09:14 -0000

SHOULD sounds good to me, with the proposed discussion.

The main point is that the invalidation behavior is optional for the sender (and even the RCaP), and therefore must be checked by the receiver. Fortunately, it is readily detectable and the requirements are well-understood.

From: david.black@emc.com [mailto:david.black@emc.com]
Sent: Monday, April 23, 2012 5:59 PM
To: Tom Talpey
Cc: storm@ietf.org
Subject: Invalidation (was RE: [storm] iSER - another try)

Tom,

> The "invalidate" form is, however, sensible to allow after carefully documenting the requirement.
> The initiator MUST verify that the appropriate memory region was invalidated, if any. However,
> whether it MUST or SHOULD perform some additional action is going to be a tough choice. A MUST
> could be too strong of a requirement, if the implementation chooses to cache or share mappings.
> On the other hand, SHOULD can introduce security vulnerabilities. Thoughts?

I think a SHOULD requirement would be ok if text is added to the security considerations section
to discuss exposure of initiator memory via STags, the initiator's responsibility to manage that
exposure via STag invalidation, and the potential security consequences of lazy invalidation that
an initiator implementation may choose to trade off against performance benefits.  This would be
in addition to new security considerations text that warns that an initiator MUST NOT rely upon
responder use of "invalidate" forms of Send (your "MUST verify" stated as a "MUST NOT" requirement).

Thanks,
--David

From: Tom Talpey [mailto:ttalpey@microsoft.com]<mailto:[mailto:ttalpey@microsoft.com]>
Sent: Monday, April 23, 2012 5:31 PM
To: Michael Ko; Black, David
Cc: storm@ietf.org<mailto:storm@ietf.org>
Subject: RE: [storm] iSER - another try

We need to separate the issues of Solicited Events from Invalidations.

I don't believe that the use of SendSE is somehow detectable with this approach. The conclusion from item ii) is that the iSER initiator SHOULD NOT arm for solicited events - it MUST be prepared for any event, whether solicited or not. This in turn implies that senders (targets) SHOULD NOT use SendSE at all - unless it's certain that *both* the RCaP and the initiator support them. This feature can neither be detected, nor is it negotiated by iSER.

The "invalidate" form is, however, sensible to allow after carefully documenting the requirement. The initiator MUST verify that the appropriate memory region was invalidated, if any. However, whether it MUST or SHOULD perform some additional action is going to be a tough choice. A MUST could be too strong of a requirement, if the implementation chooses to cache or share mappings. On the other hand, SHOULD can introduce security vulnerabilities. Thoughts?

Also,  isn't the following a new requirement on the RDMA layer?

>(lack of receiver support MUST result in an error send back to the sender).



From: storm-bounces@ietf.org<mailto:storm-bounces@ietf.org> [mailto:storm-bounces@ietf.org]<mailto:[mailto:storm-bounces@ietf.org]> On Behalf Of Michael Ko
Sent: Monday, April 23, 2012 11:55 AM
To: david.black@emc.com<mailto:david.black@emc.com>
Cc: storm@ietf.org<mailto:storm@ietf.org>
Subject: Re: [storm] iSER - another try

David,

I agree with this position.

Mike
On Mon, Apr 23, 2012 at 6:02 AM, <david.black@emc.com<mailto:david.black@emc.com>> wrote:
We don't seem to be making progress on definitive requirements that
vary by RCaP, so here's another suggestion (would be [4] if I were
numbering it ...)

It's clear that at least one implementation has diverged from the
requirements in RFC 5046 for use of SendSE, SendInv and SendInvSE.
I suggest documenting this and applying the IETF approach of "be
conservative in what you send, be liberal in what you accept",
roughly as follows:

i) Some iSER implementations have not followed RFC 5046's strict requirements
       for use of SendSE, SendInv and SendInvSE; they use Send instead.
ii) For interoperability, iSER implementations SHOULD accept and correctly
       process SendSE, SendInv and SendInvSE messages.
iii) SendSE, SendInv and SendInvSE should be regarded as optimizations or
       enhancements to the basic Send message, and their support varies by
       RCaP.  If these messages are used, the implementation SHOULD be
       capable of reverting to use of Send in order to work with a receiver
       that does not support these messages (lack of receiver support MUST
       result in an error send back to the sender).  These messages
       SHOULD NOT be used with the InfiniBand RCaP because InfiniBand does
       not require support for their functionality.
iv) New iSER implementations SHOULD use Send (and not these three additional
       messages) unless there are compelling reasons for doing otherwise
       (latter is implicit in use of "SHOULD", but worth saying explicitly,
       IMHO).
v) Some new text will be needed (including Security Considerations) to
       make it clear that STag invalidation is necessary for security
       reasons, and has to be performed independent of whether an
       Invalidate version of Send was used or not.

Is this workable?

Thanks,
--David
----------------------------------------------------
David L. Black, Distinguished Engineer
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953<tel:%2B1%20%28508%29%20293-7953>             FAX: +1 (508) 293-7786<tel:%2B1%20%28508%29%20293-7786>
david.black@emc.com<mailto:david.black@emc.com>        Mobile: +1 (978) 394-7754<tel:%2B1%20%28978%29%20394-7754>
----------------------------------------------------

_______________________________________________
storm mailing list
storm@ietf.org<mailto:storm@ietf.org>
https://www.ietf.org/mailman/listinfo/storm