Re: [storm] iSER - what to do

Michael Ko <> Tue, 26 June 2012 18:45 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 1BDDA11E80C5 for <>; Tue, 26 Jun 2012 11:45:21 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -3.598
X-Spam-Status: No, score=-3.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id Jysh4BE2QsfB for <>; Tue, 26 Jun 2012 11:45:19 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTP id A1A5611E809F for <>; Tue, 26 Jun 2012 11:45:13 -0700 (PDT)
Received: by qcmt36 with SMTP id t36so133743qcm.15 for <>; Tue, 26 Jun 2012 11:45:13 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=hfzSPI2e2om3xmF0Z/68ah7xiv4++5dnsENG6NAZvwg=; b=zmoiQcAB93UlW6qhfdDMJfiEbSnrwfjcJ6x76b2t59cckhTU9A33v1sTZObu4cNLva haH3hfZMVpQ3rhvs2RgguaN+6FfQoXeDUcQ7pkpVgiGfToiZHhN81uAp/pQfRl6fieuc XGXFR8MisrwTuy2qJy8zyFPwJ6hXcnVlaAoA2lSf1nEnMgnNio2SpwCTE1f7yuwuY3qz x17hH73f/KrPZmfqFllGyxqv5M6hN9CfT8N8X3ZSt5ypOTJqtypl+j4/o0xV7oed3QHQ COJ6F/qv74dT2o0JjI1JY29fjAOwtV9zAHWnlXHZd9at8PL2n7sZo5W8+KeTMELUdGBC yBVg==
MIME-Version: 1.0
Received: by with SMTP id n19mr7488332qct.11.1340736312898; Tue, 26 Jun 2012 11:45:12 -0700 (PDT)
Received: by with HTTP; Tue, 26 Jun 2012 11:45:12 -0700 (PDT)
In-Reply-To: <>
References: <>
Date: Tue, 26 Jun 2012 11:45:12 -0700
Message-ID: <>
From: Michael Ko <>
Content-Type: multipart/alternative; boundary=00248c769122aaf56d04c3647f0e
Subject: Re: [storm] iSER - what to do
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Storage Maintenance WG <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Tue, 26 Jun 2012 18:45:21 -0000

The last minute problem as related by Alex stems from the fact that the
iSER initiator posts its RX buffers for that connection only after the
final Login Response arrives.  During that time (after the target had sent
the Last Login Response but before
the Full Featured phase related RX-buffers are posted on the initiator
side) the target may send the first NOP-IN as it considers the connection
in Full Featured phase already and MaxOutstandingUnexpectedPDUs has been
negotiated to a non-zero value.

The current solution is for the initiator to post an extra buffer before
sending the last Login Request as the target most probably sends the next
NOP-IN only after some timeout period measuring in seconds or hundreds of
milliseconds.  But this is not a foolproof solution as the target MAY
theoretically send up to MaxOustandingUnexpectedPUDs to the initiator.

The new proposed solution is to require iSER Hello exchanges in order to
delay the onset of the iSER assisted Full Featured phase until the
iSERHelloReply PDU has arrived at the target.  This allows the initiator to
delay posting its RX buffers until after receiving the final LoginResponse
returns but before sending the iSER Hello PDU.  To support this new
proposed solution, the iSERHelloRequired key must be negotiated to Yes.
This require both sides support the new iSERHelloRequired key, leading to
the need for clarifying texts suggested by David in the spec in order to
specify what should be done otherwise.

But if we look at the current solution, the initiator is already posting an
extra buffer before sending the last Login Request.  Given that the
existing implementation is broken and has to be fixed anyway, how difficult
would it be for it to post MaxOustandingUnexpectedPDU instead of just an
extra one before sending the last Login Request?  This complies with the
spec and is guaranteed to work.  The alternative is to require the use of
and implement iSER Hello exchanges.  If the partner does not support the
new iSERHelloRequired key, the solution fails anyway.  It seems to me it
makes more sense (and a cleaner soluton besides) for the initiator to
allocate the iSER resources, i.e., MaxOustandingUnexpectedPDUs, before
sending the last Login Request as stated in the spec than requiring both
sides to support iSER Hello exchanges and negotiate the new key.


On Tue, Jun 26, 2012 at 6:16 AM, <> wrote:

> <WG chair hat on>
> The discussion about whether the buffer behavior of the Linux iSER
> implementation complies with RFC 5046 is missing at least one
> important point - to my knowledge, there are *no* iSER
> implementations that comply with RFC 5046.
> After RFC 5046 was issued, implementers looked at its requirements and
> decided to do a number of things differently - the important difference
> for this discussion is that the iSER Hello exchange is not implemented,
> even though RFC 5046 requires that it be used.  There are a number of
> other "running code" changes in the current iSER draft that reflect
> implementations and that are incompatible with RFC 5046, see Appendix A
> of the draft for details.
> The iSER work item was added to the storm WG's charter in order to produce
> an RFC specification of iSER that matches the "running code" and hence will
> interoperate with existing implementations.  Here's the text from the
> WG charter:
>        (6) iSER: Experience with Infiniband implementations suggest
>        a few minor updates to reflect what has been done in practice.
> As WG chair, I'm going to insist on the following:
> 1) The iSER draft has to match the implementations in order to be
>        published as an RFC.  Replacing RFC 5046 with another RFC that
>        won't interoperate with the implementations is pointless.
> 2) The issue encountered by the Linux iSER implementation cannot be
>        ignored by the WG.  The workaround (1 extra buffer, at most
>        one unsolicited PDU until initiator has time to deploy
>        resources) at least needs to be explained in the draft.
> 3) While requiring the iSER Hello exchange may be the proverbial
>        "right thing to do" going forward, it is necessary to provide
>        at least guidance on how to interoperate with current iSER
>        implementations that don't support the Hello exchange.
> In contrast, as WG chair, I'm not going to insist that we adopt the
> workaround as part of the standard.  For example, it's possible to
> recommend (SHOULD) use of the iSER Hello exchange (explicitly negotiated
> using the new key) and explain the workaround as what should be
> done in order to interoperate if that key results in a NotUnderstood
> response.  For this approach to the workaround, I think a lower case
> "should" may be acceptable (e.g., as opposed to a "SHOULD" or "MUST",
> but that's subject to further discussion.
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Distinguished Engineer
> EMC Corporation, 176 South St., Hopkinton, MA  01748
> +1 (508) 293-7953             FAX: +1 (508) 293-7786
>        Mobile: +1 (978) 394-7754
> ----------------------------------------------------
> _______________________________________________
> storm mailing list