Re: [storm] iSER draft

Alexander Nezhinsky <nezhinsky@gmail.com> Thu, 30 June 2011 18:07 UTC

Return-Path: <nezhinsky@gmail.com>
X-Original-To: storm@ietfa.amsl.com
Delivered-To: storm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1BA5011E823C for <storm@ietfa.amsl.com>; Thu, 30 Jun 2011 11:07:52 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.598
X-Spam-Level:
X-Spam-Status: No, score=-3.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8cfP4HUAAa3k for <storm@ietfa.amsl.com>; Thu, 30 Jun 2011 11:07:51 -0700 (PDT)
Received: from mail-yw0-f44.google.com (mail-yw0-f44.google.com [209.85.213.44]) by ietfa.amsl.com (Postfix) with ESMTP id C3CA111E8240 for <storm@ietf.org>; Thu, 30 Jun 2011 11:07:46 -0700 (PDT)
Received: by ywp31 with SMTP id 31so1261056ywp.31 for <storm@ietf.org>; Thu, 30 Jun 2011 11:07:46 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=V56kMUeXYDjUIeeM8yVTjokrNIlm4NoGQsjcH5qC6K0=; b=yEhJklZyoZ61bOiccqCPuOoncCzVVn0nWGjnBCghCg3TK0V9lnogYuMlat2eBPo+Ss S9VZyK3l+hN/rC/e9jc4AZgKG0oRqz+dNmE/0GBOYyFZDW/5oD9fvyr+ndO1dMLe+ofc 8tb0xB1MuGHIeM5P7GK/BNjCQdwIropSvbQZU=
MIME-Version: 1.0
Received: by 10.236.78.65 with SMTP id f41mr2968696yhe.233.1309457266283; Thu, 30 Jun 2011 11:07:46 -0700 (PDT)
Received: by 10.236.203.196 with HTTP; Thu, 30 Jun 2011 11:07:46 -0700 (PDT)
In-Reply-To: <971779FD6B314A40B1A3658E521C0D4A@china.huawei.com>
References: <BANLkTi=GVHUBdsZiwsRXxETd_rU=F8f85w@mail.gmail.com> <F83812DF4B59B9499C1BC978336D91745EDE207F@TK5EX14MBXC118.redmond.corp.microsoft.com> <971779FD6B314A40B1A3658E521C0D4A@china.huawei.com>
Date: Thu, 30 Jun 2011 21:07:46 +0300
Message-ID: <BANLkTik0rG2=orD0S0L3s62gBSJsLapA+w@mail.gmail.com>
From: Alexander Nezhinsky <nezhinsky@gmail.com>
To: Michael Ko <Michael@huaweisymantec.com>
Content-Type: multipart/alternative; boundary=20cf300514fa3492cb04a6f1c730
Cc: storm@ietf.org
Subject: Re: [storm] iSER draft
X-BeenThere: storm@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Storage Maintenance WG <storm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/storm>, <mailto:storm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/storm>
List-Post: <mailto:storm@ietf.org>
List-Help: <mailto:storm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/storm>, <mailto:storm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 30 Jun 2011 18:07:52 -0000

On Thu, Jun 30, 2011 at 12:26 AM, Michael Ko <Michael@huaweisymantec.com>wrote;wrote:

> **
> Tom,
>
> As originally written, the iSER spec always assumes ZBVA (zero-based
> virtual address).  But when iSER is used in conjunction with transports
> other than iWARP, there is a need to include the virtual address for the
> tagged buffer.  Such is the case for Infiniband.  Accordingly, the Virtual
> Address field has been added to the iSCSI Control Type PDUs (sec. 9.2).
> Alex pointed out that the tagged buffer should now be identified by the
> 4-tuple (STag, VA, TO, and length) instead.  This should be updated
> everywhere where the term "tagged buffer" is mentioned in the spec.  For
> example, in sec. 1.1 under Advertisement, the sentence "A typical method
> would be for the iSER Layer to embed the Tagged Buffer's STag, TO, and
> buffer length in a Send Message destined for the remote iSER Layer" should
> be modified to "A typical method would be for the iSER Layer to embed the
> Tagged Buffer's STag, VA, TO, and buffer length in a Send Message destined
> for the remote iSER Layer."  BTW, this sentence also answers your question
> as to whether VA is passed on the wire.
>
>

Agree, using VA was in the draft already, I only suggested to clarify its
use (and modes of use) from the level of definitions and up,
so that it won't "suddenly" appear in the header format description section.


> The Virtual Address is literally the virtual address of the tagged buffer.
> The tagged offset is the displacement relative to the beginning of the
> buffer starting at the virtual address.
>
> We could rephrase the definition of Virtual Address in sec. 1.1 as follows:
>
> "Virtual Address (VA) - This is the virtual address of the Tagged Buffer on
> a Node.  When set to 0, it indicates that a virtual address is not needed to
> locate the Tagged Buffer."
>
>

> Read and write buffers are identified by the combination of STag and VA.
>

OK. I tried to make the same point in the following:

> Both fields should be required for any transport. This requirement does not
> preclude
> using ZBVA where possible. When VA is not used (as in iWARP) it is set to
> 0.
>
> Below is the actually used format:
>
> struct iser_hdr {
>   uint8_t   flags;
>   uint8_t   rsvd[3];
>   uint32_t  write_stag; /* write rkey in IB */
>   uint64_t  write_va;
>   uint32_t  read_stag;  /* read rkey in IB */
>   uint64_t  read_va;
> }
>
> I would like to add that InfiniBand does have ZBVA mode. It supposedly
provides less than optimal performance with IB.
And it is not exported through the current rdma-cm API, which supports
RDMA/Ethernet as well, and is the only option
to use RDMA in user space linux apps, as of today.

So you could look at the additional fields as a kind of "reserved" in case
of standard ZBVA implementation,
used to practically support certain APIs and get enhanced performance in
some cases.

Alex