Re: [storm] DDP messages ordering

Elena Gurevich <elena.gurevich@toganetworks.com> Thu, 10 September 2015 12:02 UTC

Return-Path: <elena.gurevich@toganetworks.com>
X-Original-To: storm@ietfa.amsl.com
Delivered-To: storm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C553D1A883F for <storm@ietfa.amsl.com>; Thu, 10 Sep 2015 05:02:58 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.902
X-Spam-Level:
X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xh2Y0QMpeb79 for <storm@ietfa.amsl.com>; Thu, 10 Sep 2015 05:02:56 -0700 (PDT)
Received: from emea01-db3-obe.outbound.protection.outlook.com (mail-db3on0667.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe04::667]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C205E1B55BB for <storm@ietf.org>; Thu, 10 Sep 2015 05:02:55 -0700 (PDT)
Received: from HE1PR02MB0650.eurprd02.prod.outlook.com (10.161.117.28) by HE1PR02MB0954.eurprd02.prod.outlook.com (10.163.172.24) with Microsoft SMTP Server (TLS) id 15.1.262.15; Thu, 10 Sep 2015 12:02:35 +0000
Received: from HE1PR02MB0652.eurprd02.prod.outlook.com (10.161.118.12) by HE1PR02MB0650.eurprd02.prod.outlook.com (10.161.117.28) with Microsoft SMTP Server (TLS) id 15.1.262.15; Thu, 10 Sep 2015 12:02:33 +0000
Received: from HE1PR02MB0652.eurprd02.prod.outlook.com ([10.161.118.12]) by HE1PR02MB0652.eurprd02.prod.outlook.com ([10.161.118.12]) with mapi id 15.01.0262.011; Thu, 10 Sep 2015 12:02:33 +0000
From: Elena Gurevich <elena.gurevich@toganetworks.com>
To: Tom Talpey <tom@talpey.com>, "storm@ietf.org" <storm@ietf.org>
Thread-Topic: [storm] DDP messages ordering
Thread-Index: AdDqHUWNk/8QkGT5SXeSvBvi6thoNgAI4rqAAC+dhtAABhopgAApL59w
Date: Thu, 10 Sep 2015 12:02:32 +0000
Message-ID: <HE1PR02MB0652E769617101AEA243E918F9510@HE1PR02MB0652.eurprd02.prod.outlook.com>
References: <HE1PR02MB0652BA180545E2D5E21AB8FEF9530@HE1PR02MB0652.eurprd02.prod.outlook.com> <55EEED81.2060508@talpey.com> <HE1PR02MB06523492AC64C6ECE3BD9068F9520@HE1PR02MB0652.eurprd02.prod.outlook.com> <55F055FF.2050506@talpey.com>
In-Reply-To: <55F055FF.2050506@talpey.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is ) smtp.mailfrom=elena.gurevich@toganetworks.com;
x-originating-ip: [84.94.204.35]
x-microsoft-exchange-diagnostics: 1; HE1PR02MB0650; 5:CTa4Ygk1guYxnHSMOEoGQZqkTQD+RTyj4rrDL0+qEAIjvFsK7eBfzPLtR4tSgCUXlEQaLdHy85QX14xhS+u1PxTOHCHmibahv2v3EAtAgTvK8FQQRr/J9X5wJXCWTjOGd2uR85aFFwFS5Qoy5T9NyA==; 24:igXTlyfXjP/FLhwOFu7v1h90p6+biRGkH9lZsaYvfhYulxeE6KnIrBeBmkoDQOwIt2P45Y0tUAXj1iJ3Q29yNvnFx76DLtZml7K6NIY+IzY=; 20:GAj/5B6PiWwDd5sO5e4iCWenPWgpWh85yHu8Mp0ADkapQrG6gyBxvkoJ2hYghpI3t3FmvE09K2AXyHLxxdFEKA==
x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:HE1PR02MB0650;
x-microsoft-antispam-prvs: <HE1PR02MB065035DCB5337E928F04AF10F9510@HE1PR02MB0650.eurprd02.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:;
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(601004)(8121501046)(5005006)(3002001); SRVR:HE1PR02MB0650; BCL:0; PCL:0; RULEID:; SRVR:HE1PR02MB0650;
x-forefront-prvs: 06952FC175
x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(479174004)(377454003)(199003)(189002)(13464003)(24454002)(34854003)(42154003)(15975445007)(102836002)(40100003)(10400500002)(106356001)(62966003)(2950100001)(46102003)(66066001)(93886004)(5002640100001)(81156007)(5007970100001)(5001860100001)(2900100001)(2501003)(5001770100001)(5001960100002)(5004730100002)(5890100001)(76576001)(107886002)(189998001)(105586002)(92566002)(86362001)(5001830100001)(5001920100001)(74316001)(54356999)(77096005)(87936001)(50986999)(76176999)(19580395003)(19580405001)(68736005)(77156002)(122556002)(101416001)(5003600100002)(4001540100001)(33656002)(64706001)(11100500001)(97736004); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR02MB0650; H:HE1PR02MB0652.eurprd02.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en;
received-spf: None (protection.outlook.com: toganetworks.com does not designate permitted sender hosts)
spamdiagnosticoutput: 1:23
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Sep 2015 12:02:32.7488 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 73f7e7df-ca98-4f08-bf85-f137b447da96
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR02MB0650
X-Microsoft-Exchange-Diagnostics: 1; HE1PR02MB0954; 2:7SwO1uPCvqkdxBzARnoH9q+rHVoiyeTAePJZZqvtZ/2/NXM3VasBS168/kHUXdp0w9zBO1O0DBcV31cg9qiucBcBfuxlWgYT7f7QpHGhS+InfGlQS1LOGcHziTszrbYZTSe+YHUwCQ9408DtS1uFlgpehsChQBdAbSFljbU43/c=; 23:/E3Gtp+KTxBJPDrQPoVbECjuriYN8bwh8n98a7rlqG8YaQjpPG7LtOQG3xXgZz9t5t3s783qfYSOoOyeaCT4g7khWVTTlAmZaPfNjrJJ6MSEQchxZwvNfntJJFIwK/6CewBdU5Phttvc8RhWEhzYDJ55G3FeBxCbIVxG4+5qr0/SvcRIB6f0oXSKVLnIZF+K
X-OriginatorOrg: toganetworks.com
Archived-At: <http://mailarchive.ietf.org/arch/msg/storm/qOHQ2rytPHQxxWml3a0mkx1anS4>
Subject: Re: [storm] DDP messages ordering
X-BeenThere: storm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Storage Maintenance WG <storm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/storm>, <mailto:storm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/storm/>
List-Post: <mailto:storm@ietf.org>
List-Help: <mailto:storm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/storm>, <mailto:storm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 10 Sep 2015 12:02:59 -0000

Hello Tom,

Section 5.5 just specifies ordering rules of RDMAP requester and responder.
According to such rules RDMAP generates RDMA messages and passes them to DDP layer one by one for future processing.
RDMAP's granularity and visibility is a particular RDMA message.
DDP layer gets RDMA messages from RDMAP layer maps them one-by-one to DDP messages performs segmentation and passed them to transport layer.
So DDP segments interleaving/rules cannot be part of RDMAP , it is clearly DDP layer behavior.
And according to RFC 5041 Section 5.3 and already mentioned by you "first rule" cannot interleave DDP segments of different operations.

Best regards,
Lena

-----Original Message-----
From: Tom Talpey [mailto:tom@talpey.com]
Sent: Wednesday, September 09, 2015 6:54 PM
To: Elena Gurevich; storm@ietf.org
Subject: Re: [storm] DDP messages ordering

On 9/9/2015 9:07 AM, Elena Gurevich wrote:
> Hello,
>
> During my testing of some iWARP adaptor I discovered that RNIC interleaves DDP segments of Send and Read Response messages.
> According to your previous response this behavior is forbidden, is it ?

Not necessarily, because an RDMA Read response is special. It's injected into the DDP stream within the RNIC by RDMAP, and therefore it follows the RDMAP rules. See section 5.5 of RFC5040.

Tom.

>
> Best regards,
> Lena
>
> -----Original Message-----
> From: storm [mailto:storm-bounces@ietf.org] On Behalf Of Tom Talpey
> Sent: Tuesday, September 08, 2015 5:15 PM
> To: storm@ietf.org
> Subject: Re: [storm] DDP messages ordering
>
> On 9/8/2015 6:02 AM, Elena Gurevich wrote:
>> Hello,
>>
>> RFC 5041 section 5.3 stands that
>>
>> ----------------------------------
>>
>> 5.3 Ordering Among DDP Messages
>>
>> Messages passed through the DDP MUST conform to the ordering rules
>>
>> defined in this section.
>>
>> At the Data Source, DDP:
>>
>> * MUST transmit DDP Messages in the order they were submitted to
>>
>> the DDP layer,
>>
>> * SHOULD transmit DDP Segments within a DDP Message in increasing
>>
>> MO order for Untagged DDP Messages, and in increasing TO order
>>
>> for Tagged DDP Messages.
>>
>> ----------------------------------
>>
>> Does this mean that transmitter MUST not interleave DDM segments
>> related to consequent DDP messages ?
>
> It depends on what you mean by "the transmitter". It's certainly possible that network delivery and TCP retransmission can reorder segments. But the first rule requires that DDP not interleave two separate operations when passing them to the TCP transport.
>
> Tom.
>
> _______________________________________________
> storm mailing list
> storm@ietf.org
> https://www.ietf.org/mailman/listinfo/storm
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> ----- This email and any files transmitted and/or attachments with it
> are confidential and proprietary information of Toga Networks Ltd.,
> and intended solely for the use of the individual or entity to whom they are addressed.
> If you have received this email in error please notify the system
> manager. This message contains confidential information of Toga
> Networks Ltd., and is intended only for the individual named. If you
> are not the named addressee you should not disseminate, distribute or
> copy this e-mail. Please notify the sender immediately by e-mail if
> you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> ----
>
>
>

-------------------------------------------------------------------------------------------------------------------------------------------------
This email and any files transmitted and/or attachments with it are confidential and proprietary information of
Toga Networks Ltd., and intended solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the system manager. This message contains confidential
information of Toga Networks Ltd., and is intended only for the individual named. If you are not the named
addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately
by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not
the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on
the contents of this information is strictly prohibited.
------------------------------------------------------------------------------------------------------------------------------------------------