Re: [Ips] Data Out residual overflow/underflow handling

"Knight, Frederick" <> Fri, 21 September 2012 14:00 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 87DF621F883E for <>; Fri, 21 Sep 2012 07:00:27 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -10.598
X-Spam-Status: No, score=-10.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id O1MYTlNO3hH0 for <>; Fri, 21 Sep 2012 07:00:24 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTP id 6B30421F883C for <>; Fri, 21 Sep 2012 07:00:24 -0700 (PDT)
X-IronPort-AV: E=Sophos; i="4.80,463,1344236400"; d="scan'208,217"; a="692353614"
Received: from ([]) by with ESMTP; 21 Sep 2012 07:00:24 -0700
Received: from ( []) by (8.13.1/8.13.1/NTAP-1.6) with ESMTP id q8LE0NoM011340; Fri, 21 Sep 2012 07:00:23 -0700 (PDT)
Received: from ([]) by ([]) with mapi id 14.02.0309.002; Fri, 21 Sep 2012 07:00:22 -0700
From: "Knight, Frederick" <>
To: Paul Hughes <>, "" <>
Thread-Topic: [Ips] Data Out residual overflow/underflow handling
Thread-Index: AQHNl6g2O+NxDnTbLUmdQtQdgB+XcJeUsD4Q
Date: Fri, 21 Sep 2012 14:00:21 +0000
Message-ID: <>
References: <>
In-Reply-To: <>
Accept-Language: en-US
Content-Language: en-US
x-originating-ip: []
Content-Type: multipart/alternative; boundary="_000_FFEE311F0EC80C4EA64EC8757CAD0CD208EA1E32SACEXCMBX04PRDh_"
MIME-Version: 1.0
Subject: Re: [Ips] Data Out residual overflow/underflow handling
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IP Storage <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Fri, 21 Sep 2012 14:00:27 -0000

You have a couple of choices.

For example, SBC makes several statements about non-deterministic outcomes such as:

The device server may terminate the command before processing or after the device server has transferred some or all of the data. (walking off the end of the device)

The degree that the medium is altered by this command is vendor specific. (The format command)

But those aren't directly related.  However, as you mention, FCP provides the following 3 choice:

If the command requested that data beyond the length specified by the FCP_DL field be transferred, then the device server shall set the FCP_RESID_OVER bit (see 9.5.8) to one in the FCP_RSP IU and:

a)       process the command normally except that data beyond the FCP_DL count shall not be requested or transferred;

b)       transfer no data and return CHECK CONDITION status with the sense key set to ILLEGAL REQUEST and the additional sense code set to INVALID FIELD IN COMMAND INFORMATION UNIT; or

c)           may transfer data and return CHECK CONDITION status with the sense key set to ABORTED COMMAND and the additional sense code set to INVALID FIELD IN COMMAND INFORMATION UNIT.

Case a) and c) allow the media to be modified.  Case b) implies no media alteration (since no data is transferred).  Choice b) and c) include SCSI layer sense data, but choice a) includes only an FCP layer notification (no SCSI sense data, so a broken host (one that ignores residual unless a check condition exists) will assume success).

RFC5048 says:
   If SPDTL > EDTL for a task, iSCSI Overflow MUST be signaled in the
   SCSI Response PDU as specified in [RFC3720].  The Residual Count MUST
   be set to the numerical value of (SPDTL - EDTL).

SPDTL = the SCSI layer transfer size; EDTL = the iSCSI layer transfer size.

Since the SCSI layer transfer size in your example is 4096, and the iSCSI layer transfer size is 512, then you must report the OVERFLOW as defined in RFC3720 (bit 5 in the flags field of the SCSI response PDU).  I couldn't quickly find any suggestions in RFC3720/5048 for sense data, so I'd suggest you go with choice b) or c) that FCP is using (I'd suggest staying away from choice a) since it has bad possibilities).  At least that is something hosts should already be familiar with.

The other list you might use for these kinds of questions is the SCSI reflector:<> (T10 defines the SCSI architecture and writes the various SCSI command standards).

                Fred Knight

From: [] On Behalf Of Paul Hughes
Sent: Thursday, September 20, 2012 11:22 PM
Subject: [Ips] Data Out residual overflow/underflow handling

Not sure if this is the best place to ask, but here goes...

How should an iSCSI target (SCSI direct-access block device) handle the following scenario:

An initiator issues an iSCSI Command Request PDU containing a SCSI Write CDB with a transfer length of 1 block.  The iSCSI Command Request PDU has an Expected Data Transfer Length of 512 bytes, a Data Segment Length of 512 bytes (immediate data), and the Final flag is set.  This would be a perfectly normal single block write, except that the target's logical unit is formatted with 4096-byte block size.  So it appears the initiator is confused and sending a single 512-byte block write to a logical unit that is formatted to 4KB block size.

Here are my thoughts:

1) The target could write the 512 bytes of immediate data plus 3584 bytes (4096 minus 512) of whatever it wants to the media, and then send an iSCSI Command Response PDU with SCSI status of Good and reporting an Overflow with a residual count of 3584.  This seems to be the most correct way of handling this scenario, but it seems dangerous to allow an apparently confused initiator to essentially corrupt data on the logical unit.

2) The target could send an iSCSI Command Response PDU with SCSI status of Check Condition, with sense data of Aborted Command, Invalid Field in Command Information Unit (0x0E03).  This sense code is apparently intended for FCP (I found it mentioned in FCP-4) but it seems appropriate in this case.  Assuming the target can fail the SCSI Write command this way (or some other way), should the target also report an Underflow with a residual count of 512?

Are there any other alternatives?