Re: [storm] iSER - what to do
<david.black@emc.com> Tue, 26 June 2012 19:02 UTC
Return-Path: <david.black@emc.com>
X-Original-To: storm@ietfa.amsl.com
Delivered-To: storm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5F5B211E80CA for <storm@ietfa.amsl.com>; Tue, 26 Jun 2012 12:02:32 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.373
X-Spam-Level:
X-Spam-Status: No, score=-102.373 tagged_above=-999 required=5 tests=[AWL=0.225, BAYES_00=-2.599, HTML_MESSAGE=0.001, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F3+YUKclnEiz for <storm@ietfa.amsl.com>; Tue, 26 Jun 2012 12:02:28 -0700 (PDT)
Received: from mexforward.lss.emc.com (hop-nat-141.emc.com [168.159.213.141]) by ietfa.amsl.com (Postfix) with ESMTP id 4FBCF21F8569 for <storm@ietf.org>; Tue, 26 Jun 2012 12:02:27 -0700 (PDT)
Received: from hop04-l1d11-si02.isus.emc.com (HOP04-L1D11-SI02.isus.emc.com [10.254.111.55]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q5QJ2NhC031477 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 26 Jun 2012 15:02:24 -0400
Received: from mailhub.lss.emc.com (mailhubhoprd03.lss.emc.com [10.254.221.145]) by hop04-l1d11-si02.isus.emc.com (RSA Interceptor); Tue, 26 Jun 2012 15:02:11 -0400
Received: from mxhub21.corp.emc.com (mxhub21.corp.emc.com [128.222.70.133]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q5QJ2AHX017420; Tue, 26 Jun 2012 15:02:10 -0400
Received: from mx15a.corp.emc.com ([169.254.1.189]) by mxhub21.corp.emc.com ([128.222.70.133]) with mapi; Tue, 26 Jun 2012 15:02:10 -0400
From: david.black@emc.com
To: mkosjc@gmail.com
Date: Tue, 26 Jun 2012 15:02:08 -0400
Thread-Topic: [storm] iSER - what to do
Thread-Index: Ac1Ty+GpVpipEWmqT+25wCmH5a9MOgAASPsg
Message-ID: <8D3D17ACE214DC429325B2B98F3AE71208C14A8F@MX15A.corp.emc.com>
References: <8D3D17ACE214DC429325B2B98F3AE71208C14966@MX15A.corp.emc.com> <CAP_=6d+-VfyBOOP4pudwqZxy6dtRD=OPzeZ=W3br=KkPJGfnoQ@mail.gmail.com>
In-Reply-To: <CAP_=6d+-VfyBOOP4pudwqZxy6dtRD=OPzeZ=W3br=KkPJGfnoQ@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US
Content-Type: multipart/alternative; boundary="_000_8D3D17ACE214DC429325B2B98F3AE71208C14A8FMX15Acorpemccom_"
MIME-Version: 1.0
X-EMM-MHVC: 1
Cc: storm@ietf.org
Subject: Re: [storm] iSER - what to do
X-BeenThere: storm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Storage Maintenance WG <storm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/storm>, <mailto:storm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/storm>
List-Post: <mailto:storm@ietf.org>
List-Help: <mailto:storm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/storm>, <mailto:storm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 26 Jun 2012 19:02:32 -0000
Mike, <WG chair hat off> Thanks for responding - we will eventually get to a conclusion in this discussion :-). I'm adding two comments to try to sharpen the discussion of the current solution: > The current solution is for the initiator to post an extra buffer before > sending the last Login Request as the target most probably sends the next > NOP-IN only after some timeout period measuring in seconds or hundreds of > milliseconds. But this is not a foolproof solution as the target MAY > theoretically send up to MaxOustandingUnexpectedPUDs to the initiator. Right - my thought about how to make it fool-resistant is to require that the target MUST NOT send the next unsolicited PDU until at least 200ms after the first one or after receipt of a full-phase PDU from the initiator (that requirement could be a "SHOULD NOT" with a warning that sending sooner risks the initiator closing the RCaP connection). I pulled 200ms out of my proverbial "hat" and would like to see comments on what a good number would be for both IB and iWarp. > But if we look at the current solution, the initiator is already > posting an extra buffer before sending the last Login Request. Given > that the existing implementation is broken and has to be fixed anyway, > how difficult would it be for it to post MaxOustandingUnexpectedPDU > instead of just an extra one before sending the last Login Request? That's a question for Alexander and other implementers. My guess is that we're talking about quite a few buffers, and there may be resource management advantages to not allocating that full set of resources until the initiator is committed to the connection. Thanks, --David From: Michael Ko [mailto:mkosjc@gmail.com] Sent: Tuesday, June 26, 2012 2:45 PM To: Black, David Cc: storm@ietf.org Subject: Re: [storm] iSER - what to do The last minute problem as related by Alex stems from the fact that the iSER initiator posts its RX buffers for that connection only after the final Login Response arrives. During that time (after the target had sent the Last Login Response but before the Full Featured phase related RX-buffers are posted on the initiator side) the target may send the first NOP-IN as it considers the connection in Full Featured phase already and MaxOutstandingUnexpectedPDUs has been negotiated to a non-zero value. The current solution is for the initiator to post an extra buffer before sending the last Login Request as the target most probably sends the next NOP-IN only after some timeout period measuring in seconds or hundreds of milliseconds. But this is not a foolproof solution as the target MAY theoretically send up to MaxOustandingUnexpectedPUDs to the initiator. The new proposed solution is to require iSER Hello exchanges in order to delay the onset of the iSER assisted Full Featured phase until the iSERHelloReply PDU has arrived at the target. This allows the initiator to delay posting its RX buffers until after receiving the final LoginResponse returns but before sending the iSER Hello PDU. To support this new proposed solution, the iSERHelloRequired key must be negotiated to Yes. This require both sides support the new iSERHelloRequired key, leading to the need for clarifying texts suggested by David in the spec in order to specify what should be done otherwise. But if we look at the current solution, the initiator is already posting an extra buffer before sending the last Login Request. Given that the existing implementation is broken and has to be fixed anyway, how difficult would it be for it to post MaxOustandingUnexpectedPDU instead of just an extra one before sending the last Login Request? This complies with the spec and is guaranteed to work. The alternative is to require the use of and implement iSER Hello exchanges. If the partner does not support the new iSERHelloRequired key, the solution fails anyway. It seems to me it makes more sense (and a cleaner soluton besides) for the initiator to allocate the iSER resources, i.e., MaxOustandingUnexpectedPDUs, before sending the last Login Request as stated in the spec than requiring both sides to support iSER Hello exchanges and negotiate the new key. Mike On Tue, Jun 26, 2012 at 6:16 AM, <david.black@emc.com<mailto:david.black@emc.com>> wrote: <WG chair hat on> The discussion about whether the buffer behavior of the Linux iSER implementation complies with RFC 5046 is missing at least one important point - to my knowledge, there are *no* iSER implementations that comply with RFC 5046. After RFC 5046 was issued, implementers looked at its requirements and decided to do a number of things differently - the important difference for this discussion is that the iSER Hello exchange is not implemented, even though RFC 5046 requires that it be used. There are a number of other "running code" changes in the current iSER draft that reflect implementations and that are incompatible with RFC 5046, see Appendix A of the draft for details. The iSER work item was added to the storm WG's charter in order to produce an RFC specification of iSER that matches the "running code" and hence will interoperate with existing implementations. Here's the text from the WG charter: (6) iSER: Experience with Infiniband implementations suggest a few minor updates to reflect what has been done in practice. As WG chair, I'm going to insist on the following: 1) The iSER draft has to match the implementations in order to be published as an RFC. Replacing RFC 5046 with another RFC that won't interoperate with the implementations is pointless. 2) The issue encountered by the Linux iSER implementation cannot be ignored by the WG. The workaround (1 extra buffer, at most one unsolicited PDU until initiator has time to deploy resources) at least needs to be explained in the draft. 3) While requiring the iSER Hello exchange may be the proverbial "right thing to do" going forward, it is necessary to provide at least guidance on how to interoperate with current iSER implementations that don't support the Hello exchange. In contrast, as WG chair, I'm not going to insist that we adopt the workaround as part of the standard. For example, it's possible to recommend (SHOULD) use of the iSER Hello exchange (explicitly negotiated using the new key) and explain the workaround as what should be done in order to interoperate if that key results in a NotUnderstood response. For this approach to the workaround, I think a lower case "should" may be acceptable (e.g., as opposed to a "SHOULD" or "MUST", but that's subject to further discussion. Thanks, --David ---------------------------------------------------- David L. Black, Distinguished Engineer EMC Corporation, 176 South St., Hopkinton, MA 01748 +1 (508) 293-7953<tel:%2B1%20%28508%29%20293-7953> FAX: +1 (508) 293-7786<tel:%2B1%20%28508%29%20293-7786> david.black@emc.com<mailto:david.black@emc.com> Mobile: +1 (978) 394-7754<tel:%2B1%20%28978%29%20394-7754> ---------------------------------------------------- _______________________________________________ storm mailing list storm@ietf.org<mailto:storm@ietf.org> https://www.ietf.org/mailman/listinfo/storm
- [storm] iSER - what to do david.black
- Re: [storm] iSER - what to do Michael Ko
- Re: [storm] iSER - what to do david.black
- Re: [storm] iSER - what to do david.black
- Re: [storm] iSER - what to do Alexander Nezhinsky
- Re: [storm] iSER - what to do david.black
- Re: [storm] iSER - what to do Alexander Nezhinsky
- Re: [storm] iSER - what to do Michael Ko
- Re: [storm] iSER - what to do david.black