Re: [storm] iSER - what to do

<david.black@emc.com> Tue, 10 July 2012 00:13 UTC

Return-Path: <david.black@emc.com>
X-Original-To: storm@ietfa.amsl.com
Delivered-To: storm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B879821F8661 for <storm@ietfa.amsl.com>; Mon, 9 Jul 2012 17:13:33 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.51
X-Spam-Level:
X-Spam-Status: No, score=-102.51 tagged_above=-999 required=5 tests=[AWL=0.088, BAYES_00=-2.599, HTML_MESSAGE=0.001, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zB231aq08ku4 for <storm@ietfa.amsl.com>; Mon, 9 Jul 2012 17:13:28 -0700 (PDT)
Received: from mexforward.lss.emc.com (hop-nat-141.emc.com [168.159.213.141]) by ietfa.amsl.com (Postfix) with ESMTP id 248A121F8643 for <storm@ietf.org>; Mon, 9 Jul 2012 17:13:18 -0700 (PDT)
Received: from hop04-l1d11-si01.isus.emc.com (HOP04-L1D11-SI01.isus.emc.com [10.254.111.54]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q6A0DiPH017711 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for <storm@ietf.org>; Mon, 9 Jul 2012 20:13:44 -0400
Received: from mailhub.lss.emc.com (mailhubhoprd01.lss.emc.com [10.254.221.251]) by hop04-l1d11-si01.isus.emc.com (RSA Interceptor) for <storm@ietf.org>; Mon, 9 Jul 2012 20:13:29 -0400
Received: from mxhub28.corp.emc.com (mxhub28.corp.emc.com [10.254.110.184]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q6A0DSRA004531 for <storm@ietf.org>; Mon, 9 Jul 2012 20:13:28 -0400
Received: from mx15a.corp.emc.com ([169.254.1.189]) by mxhub28.corp.emc.com ([10.254.110.184]) with mapi; Mon, 9 Jul 2012 20:13:27 -0400
From: <david.black@emc.com>
To: <storm@ietf.org>
Date: Mon, 9 Jul 2012 20:13:26 -0400
Thread-Topic: [storm] iSER - what to do
Thread-Index: Ac1Ty+GpVpipEWmqT+25wCmH5a9MOgAASPsgApjLkWA=
Message-ID: <8D3D17ACE214DC429325B2B98F3AE71208D3B0CA@MX15A.corp.emc.com>
References: <8D3D17ACE214DC429325B2B98F3AE71208C14966@MX15A.corp.emc.com> <CAP_=6d+-VfyBOOP4pudwqZxy6dtRD=OPzeZ=W3br=KkPJGfnoQ@mail.gmail.com> <8D3D17ACE214DC429325B2B98F3AE71208C14A8F@MX15A.corp.emc.com>
In-Reply-To: <8D3D17ACE214DC429325B2B98F3AE71208C14A8F@MX15A.corp.emc.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US
Content-Type: multipart/alternative; boundary="_000_8D3D17ACE214DC429325B2B98F3AE71208D3B0CAMX15Acorpemccom_"
MIME-Version: 1.0
X-EMM-MHVC: 1
Subject: Re: [storm] iSER - what to do
X-BeenThere: storm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Storage Maintenance WG <storm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/storm>, <mailto:storm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/storm>
List-Post: <mailto:storm@ietf.org>
List-Help: <mailto:storm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/storm>, <mailto:storm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Jul 2012 00:13:33 -0000

<WG chair hat on>

I think we have a couple of choices.

- A revised draft this week as outlined in this thread below.
- Pick up discussion again in August or September after the Vancouver meetings.

At this point, I think we're headed for August or September.

Thanks,
--David

From: storm-bounces@ietf.org [mailto:storm-bounces@ietf.org] On Behalf Of david.black@emc.com
Sent: Tuesday, June 26, 2012 3:02 PM
To: mkosjc@gmail.com
Cc: storm@ietf.org
Subject: Re: [storm] iSER - what to do

Mike,

<WG chair hat off>

Thanks for responding - we will eventually get to a conclusion in this discussion :-).

I'm adding two comments to try to sharpen the discussion of the current solution:

> The current solution is for the initiator to post an extra buffer before
> sending the last Login Request as the target most probably sends the next
> NOP-IN only after some timeout period measuring in seconds or hundreds of
> milliseconds.  But this is not a foolproof solution as the target MAY
> theoretically send up to MaxOustandingUnexpectedPUDs to the initiator.
Right - my thought about how to make it fool-resistant is to require that
the target MUST NOT send the next unsolicited PDU until at least 200ms
after the first one or after receipt of a full-phase PDU from the initiator
(that requirement could be a "SHOULD NOT" with a warning that sending
sooner risks the initiator closing the RCaP connection).  I pulled 200ms
out of my proverbial "hat" and would like to see comments on what a
good number would be for both IB and iWarp.

> But if we look at the current solution, the initiator is already
> posting an extra buffer before sending the last Login Request.  Given
> that the existing implementation is broken and has to be fixed anyway,
> how difficult would it be for it to post MaxOustandingUnexpectedPDU
> instead of just an extra one before sending the last Login Request?

That's a question for Alexander and other implementers.  My guess is that
we're talking about quite a few buffers, and there may be resource management
advantages to not allocating that full set of resources until the initiator
is committed to the connection.

Thanks,
--David

From: Michael Ko [mailto:mkosjc@gmail.com]
Sent: Tuesday, June 26, 2012 2:45 PM
To: Black, David
Cc: storm@ietf.org
Subject: Re: [storm] iSER - what to do

The last minute problem as related by Alex stems from the fact that the iSER initiator posts its RX buffers for that connection only after the final Login Response arrives.  During that time (after the target had sent the Last Login Response but before
the Full Featured phase related RX-buffers are posted on the initiator side) the target may send the first NOP-IN as it considers the connection in Full Featured phase already and MaxOutstandingUnexpectedPDUs has been negotiated to a non-zero value.

The current solution is for the initiator to post an extra buffer before sending the last Login Request as the target most probably sends the next NOP-IN only after some timeout period measuring in seconds or hundreds of milliseconds.  But this is not a foolproof solution as the target MAY theoretically send up to MaxOustandingUnexpectedPUDs to the initiator.

The new proposed solution is to require iSER Hello exchanges in order to delay the onset of the iSER assisted Full Featured phase until the iSERHelloReply PDU has arrived at the target.  This allows the initiator to delay posting its RX buffers until after receiving the final LoginResponse returns but before sending the iSER Hello PDU.  To support this new proposed solution, the iSERHelloRequired key must be negotiated to Yes.  This require both sides support the new iSERHelloRequired key, leading to the need for clarifying texts suggested by David in the spec in order to specify what should be done otherwise.

But if we look at the current solution, the initiator is already posting an extra buffer before sending the last Login Request.  Given that the existing implementation is broken and has to be fixed anyway, how difficult would it be for it to post MaxOustandingUnexpectedPDU instead of just an extra one before sending the last Login Request?  This complies with the spec and is guaranteed to work.  The alternative is to require the use of and implement iSER Hello exchanges.  If the partner does not support the new iSERHelloRequired key, the solution fails anyway.  It seems to me it makes more sense (and a cleaner soluton besides) for the initiator to allocate the iSER resources, i.e., MaxOustandingUnexpectedPDUs, before sending the last Login Request as stated in the spec than requiring both sides to support iSER Hello exchanges and negotiate the new key.

Mike
On Tue, Jun 26, 2012 at 6:16 AM, <david.black@emc.com<mailto:david.black@emc.com>> wrote:
<WG chair hat on>

The discussion about whether the buffer behavior of the Linux iSER
implementation complies with RFC 5046 is missing at least one
important point - to my knowledge, there are *no* iSER
implementations that comply with RFC 5046.

After RFC 5046 was issued, implementers looked at its requirements and
decided to do a number of things differently - the important difference
for this discussion is that the iSER Hello exchange is not implemented,
even though RFC 5046 requires that it be used.  There are a number of
other "running code" changes in the current iSER draft that reflect
implementations and that are incompatible with RFC 5046, see Appendix A
of the draft for details.

The iSER work item was added to the storm WG's charter in order to produce
an RFC specification of iSER that matches the "running code" and hence will
interoperate with existing implementations.  Here's the text from the
WG charter:

       (6) iSER: Experience with Infiniband implementations suggest
       a few minor updates to reflect what has been done in practice.

As WG chair, I'm going to insist on the following:
1) The iSER draft has to match the implementations in order to be
       published as an RFC.  Replacing RFC 5046 with another RFC that
       won't interoperate with the implementations is pointless.
2) The issue encountered by the Linux iSER implementation cannot be
       ignored by the WG.  The workaround (1 extra buffer, at most
       one unsolicited PDU until initiator has time to deploy
       resources) at least needs to be explained in the draft.
3) While requiring the iSER Hello exchange may be the proverbial
       "right thing to do" going forward, it is necessary to provide
       at least guidance on how to interoperate with current iSER
       implementations that don't support the Hello exchange.

In contrast, as WG chair, I'm not going to insist that we adopt the
workaround as part of the standard.  For example, it's possible to
recommend (SHOULD) use of the iSER Hello exchange (explicitly negotiated
using the new key) and explain the workaround as what should be
done in order to interoperate if that key results in a NotUnderstood
response.  For this approach to the workaround, I think a lower case
"should" may be acceptable (e.g., as opposed to a "SHOULD" or "MUST",
but that's subject to further discussion.

Thanks,
--David
----------------------------------------------------
David L. Black, Distinguished Engineer
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953<tel:%2B1%20%28508%29%20293-7953>             FAX: +1 (508) 293-7786<tel:%2B1%20%28508%29%20293-7786>
david.black@emc.com<mailto:david.black@emc.com>        Mobile: +1 (978) 394-7754<tel:%2B1%20%28978%29%20394-7754>
----------------------------------------------------


_______________________________________________
storm mailing list
storm@ietf.org<mailto:storm@ietf.org>
https://www.ietf.org/mailman/listinfo/storm