Re: [storm] iSER - what to do

<david.black@emc.com> Tue, 26 June 2012 19:02 UTC

Return-Path: <david.black@emc.com>
X-Original-To: storm@ietfa.amsl.com
Delivered-To: storm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5F5B211E80CA for <storm@ietfa.amsl.com>; Tue, 26 Jun 2012 12:02:32 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.373
X-Spam-Level:
X-Spam-Status: No, score=-102.373 tagged_above=-999 required=5 tests=[AWL=0.225, BAYES_00=-2.599, HTML_MESSAGE=0.001, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F3+YUKclnEiz for <storm@ietfa.amsl.com>; Tue, 26 Jun 2012 12:02:28 -0700 (PDT)
Received: from mexforward.lss.emc.com (hop-nat-141.emc.com [168.159.213.141]) by ietfa.amsl.com (Postfix) with ESMTP id 4FBCF21F8569 for <storm@ietf.org>; Tue, 26 Jun 2012 12:02:27 -0700 (PDT)
Received: from hop04-l1d11-si02.isus.emc.com (HOP04-L1D11-SI02.isus.emc.com [10.254.111.55]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q5QJ2NhC031477 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 26 Jun 2012 15:02:24 -0400
Received: from mailhub.lss.emc.com (mailhubhoprd03.lss.emc.com [10.254.221.145]) by hop04-l1d11-si02.isus.emc.com (RSA Interceptor); Tue, 26 Jun 2012 15:02:11 -0400
Received: from mxhub21.corp.emc.com (mxhub21.corp.emc.com [128.222.70.133]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q5QJ2AHX017420; Tue, 26 Jun 2012 15:02:10 -0400
Received: from mx15a.corp.emc.com ([169.254.1.189]) by mxhub21.corp.emc.com ([128.222.70.133]) with mapi; Tue, 26 Jun 2012 15:02:10 -0400
From: <david.black@emc.com>
To: <mkosjc@gmail.com>
Date: Tue, 26 Jun 2012 15:02:08 -0400
Thread-Topic: [storm] iSER - what to do
Thread-Index: Ac1Ty+GpVpipEWmqT+25wCmH5a9MOgAASPsg
Message-ID: <8D3D17ACE214DC429325B2B98F3AE71208C14A8F@MX15A.corp.emc.com>
References: <8D3D17ACE214DC429325B2B98F3AE71208C14966@MX15A.corp.emc.com> <CAP_=6d+-VfyBOOP4pudwqZxy6dtRD=OPzeZ=W3br=KkPJGfnoQ@mail.gmail.com>
In-Reply-To: <CAP_=6d+-VfyBOOP4pudwqZxy6dtRD=OPzeZ=W3br=KkPJGfnoQ@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US
Content-Type: multipart/alternative; boundary="_000_8D3D17ACE214DC429325B2B98F3AE71208C14A8FMX15Acorpemccom_"
MIME-Version: 1.0
X-EMM-MHVC: 1
Cc: storm@ietf.org
Subject: Re: [storm] iSER - what to do
X-BeenThere: storm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Storage Maintenance WG <storm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/storm>, <mailto:storm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/storm>
List-Post: <mailto:storm@ietf.org>
List-Help: <mailto:storm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/storm>, <mailto:storm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 26 Jun 2012 19:02:32 -0000

Mike,

<WG chair hat off>

Thanks for responding - we will eventually get to a conclusion in this discussion :-).

I'm adding two comments to try to sharpen the discussion of the current solution:

> The current solution is for the initiator to post an extra buffer before
> sending the last Login Request as the target most probably sends the next
> NOP-IN only after some timeout period measuring in seconds or hundreds of
> milliseconds.  But this is not a foolproof solution as the target MAY
> theoretically send up to MaxOustandingUnexpectedPUDs to the initiator.

Right - my thought about how to make it fool-resistant is to require that
the target MUST NOT send the next unsolicited PDU until at least 200ms
after the first one or after receipt of a full-phase PDU from the initiator
(that requirement could be a "SHOULD NOT" with a warning that sending
sooner risks the initiator closing the RCaP connection).  I pulled 200ms
out of my proverbial "hat" and would like to see comments on what a
good number would be for both IB and iWarp.

> But if we look at the current solution, the initiator is already
> posting an extra buffer before sending the last Login Request.  Given
> that the existing implementation is broken and has to be fixed anyway,
> how difficult would it be for it to post MaxOustandingUnexpectedPDU
> instead of just an extra one before sending the last Login Request?

That's a question for Alexander and other implementers.  My guess is that
we're talking about quite a few buffers, and there may be resource management
advantages to not allocating that full set of resources until the initiator
is committed to the connection.

Thanks,
--David

From: Michael Ko [mailto:mkosjc@gmail.com]
Sent: Tuesday, June 26, 2012 2:45 PM
To: Black, David
Cc: storm@ietf.org
Subject: Re: [storm] iSER - what to do

The last minute problem as related by Alex stems from the fact that the iSER initiator posts its RX buffers for that connection only after the final Login Response arrives.  During that time (after the target had sent the Last Login Response but before
the Full Featured phase related RX-buffers are posted on the initiator side) the target may send the first NOP-IN as it considers the connection in Full Featured phase already and MaxOutstandingUnexpectedPDUs has been negotiated to a non-zero value.

The current solution is for the initiator to post an extra buffer before sending the last Login Request as the target most probably sends the next NOP-IN only after some timeout period measuring in seconds or hundreds of milliseconds.  But this is not a foolproof solution as the target MAY theoretically send up to MaxOustandingUnexpectedPUDs to the initiator.

The new proposed solution is to require iSER Hello exchanges in order to delay the onset of the iSER assisted Full Featured phase until the iSERHelloReply PDU has arrived at the target.  This allows the initiator to delay posting its RX buffers until after receiving the final LoginResponse returns but before sending the iSER Hello PDU.  To support this new proposed solution, the iSERHelloRequired key must be negotiated to Yes.  This require both sides support the new iSERHelloRequired key, leading to the need for clarifying texts suggested by David in the spec in order to specify what should be done otherwise.

But if we look at the current solution, the initiator is already posting an extra buffer before sending the last Login Request.  Given that the existing implementation is broken and has to be fixed anyway, how difficult would it be for it to post MaxOustandingUnexpectedPDU instead of just an extra one before sending the last Login Request?  This complies with the spec and is guaranteed to work.  The alternative is to require the use of and implement iSER Hello exchanges.  If the partner does not support the new iSERHelloRequired key, the solution fails anyway.  It seems to me it makes more sense (and a cleaner soluton besides) for the initiator to allocate the iSER resources, i.e., MaxOustandingUnexpectedPDUs, before sending the last Login Request as stated in the spec than requiring both sides to support iSER Hello exchanges and negotiate the new key.

Mike
On Tue, Jun 26, 2012 at 6:16 AM, <david.black@emc.com<mailto:david.black@emc.com>> wrote:
<WG chair hat on>

The discussion about whether the buffer behavior of the Linux iSER
implementation complies with RFC 5046 is missing at least one
important point - to my knowledge, there are *no* iSER
implementations that comply with RFC 5046.

After RFC 5046 was issued, implementers looked at its requirements and
decided to do a number of things differently - the important difference
for this discussion is that the iSER Hello exchange is not implemented,
even though RFC 5046 requires that it be used.  There are a number of
other "running code" changes in the current iSER draft that reflect
implementations and that are incompatible with RFC 5046, see Appendix A
of the draft for details.

The iSER work item was added to the storm WG's charter in order to produce
an RFC specification of iSER that matches the "running code" and hence will
interoperate with existing implementations.  Here's the text from the
WG charter:

       (6) iSER: Experience with Infiniband implementations suggest
       a few minor updates to reflect what has been done in practice.

As WG chair, I'm going to insist on the following:
1) The iSER draft has to match the implementations in order to be
       published as an RFC.  Replacing RFC 5046 with another RFC that
       won't interoperate with the implementations is pointless.
2) The issue encountered by the Linux iSER implementation cannot be
       ignored by the WG.  The workaround (1 extra buffer, at most
       one unsolicited PDU until initiator has time to deploy
       resources) at least needs to be explained in the draft.
3) While requiring the iSER Hello exchange may be the proverbial
       "right thing to do" going forward, it is necessary to provide
       at least guidance on how to interoperate with current iSER
       implementations that don't support the Hello exchange.

In contrast, as WG chair, I'm not going to insist that we adopt the
workaround as part of the standard.  For example, it's possible to
recommend (SHOULD) use of the iSER Hello exchange (explicitly negotiated
using the new key) and explain the workaround as what should be
done in order to interoperate if that key results in a NotUnderstood
response.  For this approach to the workaround, I think a lower case
"should" may be acceptable (e.g., as opposed to a "SHOULD" or "MUST",
but that's subject to further discussion.

Thanks,
--David
----------------------------------------------------
David L. Black, Distinguished Engineer
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953<tel:%2B1%20%28508%29%20293-7953>             FAX: +1 (508) 293-7786<tel:%2B1%20%28508%29%20293-7786>
david.black@emc.com<mailto:david.black@emc.com>        Mobile: +1 (978) 394-7754<tel:%2B1%20%28978%29%20394-7754>
----------------------------------------------------


_______________________________________________
storm mailing list
storm@ietf.org<mailto:storm@ietf.org>
https://www.ietf.org/mailman/listinfo/storm