Re: [6lo] Mirja Kühlewind's Discuss on draft-ietf-6lo-fragment-recovery-13: (with DISCUSS and COMMENT)

"Pascal Thubert (pthubert)" <pthubert@cisco.com> Fri, 20 March 2020 16:17 UTC

Return-Path: <pthubert@cisco.com>
X-Original-To: 6lo@ietfa.amsl.com
Delivered-To: 6lo@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 607703A0C2D; Fri, 20 Mar 2020 09:17:19 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.589
X-Spam-Level:
X-Spam-Status: No, score=-9.589 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, T_SPF_TEMPERROR=0.01, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com header.b=IxDtxPXe; dkim=pass (1024-bit key) header.d=cisco.onmicrosoft.com header.b=BDlFer/x
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TVD77Z9XhJCn; Fri, 20 Mar 2020 09:17:12 -0700 (PDT)
Received: from rcdn-iport-8.cisco.com (rcdn-iport-8.cisco.com [173.37.86.79]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1DAA93A0C0B; Fri, 20 Mar 2020 09:15:24 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=6688; q=dns/txt; s=iport; t=1584720924; x=1585930524; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=zen0QwRE/p5fOfgp5RAUZkOR7+ftdbpvAHIuLWp8i4A=; b=IxDtxPXefWg/PMOSG1yX4v2JkKgHl2ZY+z0tBg/FgXIKSL/sn0BXCgFI 54gLw7WoJOkt8D4Sss2Q2gofzDqZS2s7oZkjfG0rqZVViXdvG6/siaMBJ JECpQgKrCvgtdOhr9xDo/F0aQ91xfINTn60ry9Z2uin3SjgAPukLOcayX 4=;
IronPort-PHdr: =?us-ascii?q?9a23=3AnnMRdRxANp4QJ5jXCy+N+z0EezQntrPoPwUc9p?= =?us-ascii?q?sgjfdUf7+++4j5YhWN/u1j2VnOW4iTq+lJjebbqejBYSQB+t7A1RJKa5lQT1?= =?us-ascii?q?kAgMQSkRYnBZudFU3mJvPwcwQxHd9JUxlu+HToeUU=3D?=
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0D9CQCx6nRe/4oNJK1mHgELHINPJCw?= =?us-ascii?q?FgUQgBAsqhBiDRQOKcE6CEZgcgUKBEANUCQEBAQwBAS0CBAEBhEQCF4INJDg?= =?us-ascii?q?TAgMBAQsBAQUBAQECAQUEbYVWDIVjAQEBAQIBEhERDAEBJQYMAQQLAgEGAg4?= =?us-ascii?q?MAiYCAgIwFQULAgQODRqFUAMOIAGRNJBnAoE5iGJ1gTKCfwEBBYULGIIMCYE?= =?us-ascii?q?OKoUghw8agUE/gRFHgk0+hCMPG4MPMoIskDg7n1gKgjyXG4JLiCqMDIRUg1G?= =?us-ascii?q?nHwIEAgQFAg4BAQWBaSKBWHAVgydQGA2OHTiDO4pVdIEpiyeCQgEB?=
X-IronPort-AV: E=Sophos;i="5.72,285,1580774400"; d="scan'208";a="742296359"
Received: from alln-core-5.cisco.com ([173.36.13.138]) by rcdn-iport-8.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA; 20 Mar 2020 16:15:18 +0000
Received: from XCH-ALN-001.cisco.com (xch-aln-001.cisco.com [173.36.7.11]) by alln-core-5.cisco.com (8.15.2/8.15.2) with ESMTPS id 02KGFHJS000400 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=FAIL); Fri, 20 Mar 2020 16:15:18 GMT
Received: from xhs-rcd-001.cisco.com (173.37.227.246) by XCH-ALN-001.cisco.com (173.36.7.11) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Fri, 20 Mar 2020 11:15:17 -0500
Received: from xhs-rcd-003.cisco.com (173.37.227.248) by xhs-rcd-001.cisco.com (173.37.227.246) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Fri, 20 Mar 2020 11:15:17 -0500
Received: from NAM10-BN7-obe.outbound.protection.outlook.com (72.163.14.9) by xhs-rcd-003.cisco.com (173.37.227.248) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Fri, 20 Mar 2020 11:15:17 -0500
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; =?utf-8?q?b=3DlwqDNTnzfbmKVfkqma82NSaQbgnoyt9NOtMUzPj0Zrj5ocmlb6toVooGl6mbi?= =?utf-8?q?uMS0b/1hMtVrVfJHso4f0sMyd3/rceMveaKCpuG4C9ZRFJkpcGZv5T8dNd3vue8YQ?= =?utf-8?q?EJIznzPYw7Y9d+8rjqYOcvSqBV55ZcVCE2o8Z0L6tu7loozX3x65/l5zfnYwBlgeH?= =?utf-8?q?wchIfiedgm1hR5+yaDiKtXRqY1SDxQmTXPJFBnh158FVBeSn/4qk2t1eFQBU/ZHEM?= =?utf-8?q?lgRDLxmXObm+S9QuA6HistAi5/VihM43bd5XJHPghvCHTWw6uWUNoGl2ZCT2HYpPn?= =?utf-8?q?ry2/lrthMhl/fVS+EGjZg=3D=3D?=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; =?utf-8?q?h=3DFrom=3ADate=3ASubject=3AMessage-ID=3ACont?= =?utf-8?q?ent-Type=3AMIME-Version=3AX-MS-Exchange-SenderADCheck=3B?= =?utf-8?q?bh=3Dzen0QwRE/p5fOfgp5RAUZkOR7+ftdbpvAHIuLWp8i4A=3D=3B_b=3DS/b323?= =?utf-8?q?a3G5HIORqjJZY8xljvaTjkN/NXckMfpyjT3qJFC/khkRSuWQjpOjuFoaAnZq24UJS?= =?utf-8?q?mSXjD4I2XE+y/f7GVZ8BKiJgjyY+RGIOo+JKAXx8WDu5fkTLl7LUkjryOjDeszlxP?= =?utf-8?q?wIngZfa5oP1d9vnXap+zwAvk3XjmlHadVWgaF7YlibvhTQgmCiBOVSclOs0byXN1z?= =?utf-8?q?9Ey52QgwQeMxOj6tg8WBqI+fR21xfh2PMn7evWK639fOT0NW9I50yCGdS5cLKIlSf?= =?utf-8?q?LtSrwlQlxjVx/6NtYV+cpJO1I4jNa9rBgcjSMESSM/1qN0AyU57bkQWmff1Pftrzv?= =?utf-8?q?CSaOVL7DMwQ=3D=3D?=
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cisco.com; dmarc=pass action=none header.from=cisco.com; dkim=pass header.d=cisco.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.onmicrosoft.com; s=selector2-cisco-onmicrosoft-com; =?utf-8?q?h=3DFrom=3ADate=3ASubject=3AM?= =?utf-8?q?essage-ID=3AContent-Type=3AMIME-Version=3AX-MS-Exchange-SenderADC?= =?utf-8?q?heck=3B_bh=3Dzen0QwRE/p5fOfgp5RAUZkOR7+ftdbpvAHIuLWp8i4A=3D=3B_b?= =?utf-8?q?=3DBDlFer/xQHmiKPe7lvagM5rMfAPybYyHsTLUrWUa4WnoRYzzXggA4JJdod859N?= =?utf-8?q?La35h0mq9bd6vpvpSCp+4h3sC89LKm3P4hQ/umUIMd/CvQ5+CK7RtZoQv9qfmbET2?= =?utf-8?q?XtplqIo1GmrBlWskkXx58ynihDQdnlLrGJRfDbyRq4Dc=3D?=
Received: from MN2PR11MB3565.namprd11.prod.outlook.com (2603:10b6:208:ea::31) by MN2PR11MB4400.namprd11.prod.outlook.com (2603:10b6:208:18c::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2814.14; Fri, 20 Mar 2020 16:15:16 +0000
Received: from MN2PR11MB3565.namprd11.prod.outlook.com ([fe80::113b:3127:ef12:ea7]) by MN2PR11MB3565.namprd11.prod.outlook.com ([fe80::113b:3127:ef12:ea7%7]) with mapi id 15.20.2814.021; Fri, 20 Mar 2020 16:15:16 +0000
From: "Pascal Thubert (pthubert)" <pthubert@cisco.com>
To: Mirja Kuehlewind <ietf@kuehlewind.net>
CC: "6lo-chairs@ietf.org" <6lo-chairs@ietf.org>, The IESG <iesg@ietf.org>, "6lo@ietf.org" <6lo@ietf.org>, "draft-ietf-6lo-fragment-recovery@ietf.org" <draft-ietf-6lo-fragment-recovery@ietf.org>, Carles Gomez <carlesgo@entel.upc.edu>, Martin Duke <martin.h.duke@gmail.com>, "Benjamin Kaduk" <kaduk@mit.edu>
Thread-Topic: Mirja =?utf-8?q?K=C3=BChlewind=27s_Discuss_on_draft-ietf-6lo-f?= =?utf-8?q?ragment-recovery-13=3A?= (with DISCUSS and COMMENT)
Thread-Index: AQHV5yxvQMp3kJRXD0SmWqJXtHBMvqg7nFTwgBQ3dgCAABdRMIAAcc0AgAAZUYKAATf0AIAAEAyA
Date: Fri, 20 Mar 2020 16:15:05 +0000
Deferred-Delivery: Fri, 20 Mar 2020 16:15:02 +0000
Message-ID: =?utf-8?q?=3CMN2PR11MB3565B3E9A05CF5D10D125407D8F50=40MN2PR11MB3?= =?utf-8?q?565=2Enamprd11=2Eprod=2Eoutlook=2Ecom=3E?=
References: <158212059997.17584.9409485384556514167.idtracker@ietfa.amsl.com> =?utf-8?q?=3CMN2PR11MB356540107CC4FC7F9CB9F412D8E30=40MN2PR11MB3565=2Enampr?= =?utf-8?q?d11=2Eprod=2Eoutlook=2Ecom=3E?= =?utf-8?q?=3C7D77F80D-9D2C-4AF7-AB7E-4EDF58A9258A=40kuehlewind=2Enet=3E_=3C?= =?utf-8?q?MN2PR11MB3565C75A4707268047E1F90ED8F40=40MN2PR11MB3565=2Enamprd11?= =?utf-8?q?=2Eprod=2Eoutlook=2Ecom=3E?= <9823A19B-CA7D-47D1-92FE-8AA436240BD4@kuehlewind.net> <C3771F28-264D-4D0D-9864-D40711EBA0FD@cisco.com> <D3F4ECB5-3E1D-48EF-83D9-71F3F14629C4@kuehlewind.net>
In-Reply-To: <D3F4ECB5-3E1D-48EF-83D9-71F3F14629C4@kuehlewind.net>
Accept-Language: fr-FR, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is ) smtp.mailfrom=pthubert@cisco.com;
x-originating-ip: [2001:420:c0c0:1006::22c]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 05e03431-8716-4700-2c94-08d7cce9e0a3
x-ms-traffictypediagnostic: MN2PR11MB4400:
x-microsoft-antispam-prvs: =?utf-8?q?=3CMN2PR11MB440020BDAB28BB71F9CA24C3D8F?= =?utf-8?q?50=40MN2PR11MB4400=2Enamprd11=2Eprod=2Eoutlook=2Ecom=3E?=
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-forefront-prvs: 03484C0ABF
x-forefront-antispam-report: SFV:NSPM; =?utf-8?q?SFS=3A=2810009020=29=284636?= =?utf-8?b?MDA5KSgzNzYwMDIpKDM0NjAwMikoMzk2MDAzKSgzOTg2MDQwMDAwMikoMTM2?= =?utf-8?b?MDAzKSgzNjYwMDQpKDE5OTAwNCkoNjY1NTYwMDgpKDY2NDc2MDA3KSg3Njk2?= =?utf-8?q?005=29=28224303003=29=2864756008=29=2866446008=29=2854906003=29?= =?utf-8?q?=28316002=29=289686003=29=2855016002=29=2866946007=29=28478600001?= =?utf-8?b?KSg4NjM2MjAwMSkoNDMyNjAwOCkoNjY2NjAwNCkoNzYxMTYwMDYpKDU2NjAz?= =?utf-8?b?MDAwMDIpKDgxMTY2MDA2KSg4MTE1NjAxNCkoODkzNjAwMikoMTg2MDAzKSg1?= =?utf-8?q?2536014=29=282906002=29=2833656002=29=286916009=29=2871200400001?= =?utf-8?q?=29=286506007=29=3B?= DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR11MB4400; H:MN2PR11MB3565.namprd11.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1;
received-spf: None (protection.outlook.com: cisco.com does not designate permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: =?utf-8?q?2ueQbD9/Sp5S3vmEL8qPQcq6EYjGz4C?= =?utf-8?q?yheMEbLkjhiRLfqBIMO8QOtX4RWZ6WrbW5C5tXutZeRtNi3hCz4vNPyTW7wSOhwP9?= =?utf-8?q?ksvu72H5eyE+3Cj2P0hV62CdN9/Rc8Eu41bkmksgNq2JICaPC8uDkISs150KCZwau?= =?utf-8?q?uKfw8PiG57ZWkFnm2/aH22PvKbFAkGst4kSTqv6VFT90Z/bvNLzMmoyriJSiG3Or1?= =?utf-8?q?sRmK6kGLaBq31qnz0gK7ROdacRkLvyGoLSG+hm70SOupJ9dyFhjZEk/8gKiaBbdon?= =?utf-8?q?dS56XBRF/smod86m9+OegS15Su/9hTo2jzCTeQC1Z3rl4qOgRl1kPLpI9xTH1HTVB?= =?utf-8?q?RUbHMDq7FzpaqrRjZe3G/bGilb1CNvPYucy4CDlR42jRWFRtAh1SJ9lSweIQVQuVn?= =?utf-8?q?SbXN3WY2qdF0ZU2Xhns67MEV9JJ?=
x-ms-exchange-antispam-messagedata: =?utf-8?q?l9FyU724FPGWPS9DnFWqtrBfr8+5bz?= =?utf-8?q?tZwYazm2AASqkBUOTdhB0L15dY//YNhNTsd+ysVmZZ9xRthKyB+V0DhAib6Pns8f1?= =?utf-8?q?YqoZ+2tZ7jot9wcUehnmn+Buq0G62aJ0nZO2VMsXJDvuATbio/HLQ/gpSLNreRTOo?= =?utf-8?q?JsPIKU8v0v8=3D?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 05e03431-8716-4700-2c94-08d7cce9e0a3
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Mar 2020 16:15:16.1872 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 5ae1af62-9505-4097-a69a-c1553ef7840e
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: =?utf-8?q?hUajxTjXNaWi/Vvo7a/O+?= =?utf-8?q?Rp7WBLuZOtFFl3ZhP3+TbswoCWd4ib2NgHJfr2TD3fiS1kNOpo3FQHvowg1RSwaMg?= =?utf-8?q?=3D=3D?=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB4400
X-OriginatorOrg: cisco.com
X-Outbound-SMTP-Client: 173.36.7.11, xch-aln-001.cisco.com
X-Outbound-Node: alln-core-5.cisco.com
Archived-At: <https://mailarchive.ietf.org/arch/msg/6lo/I_45-VGZBJ5xpoXXwUdr8U0Tj3c>
Subject: Re: [6lo] =?utf-8?q?Mirja_K=C3=BChlewind=27s_Discuss_on_draft-ietf-6?= =?utf-8?q?lo-fragment-recovery-13=3A_=28with_DISCUSS_and_COMMENT=29?=
X-BeenThere: 6lo@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Mailing list for the 6lo WG for Internet Area issues in IPv6 over constrained node networks." <6lo.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/6lo>, <mailto:6lo-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/6lo/>
List-Post: <mailto:6lo@ietf.org>
List-Help: <mailto:6lo-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/6lo>, <mailto:6lo-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 20 Mar 2020 16:17:20 -0000

Hello Mirja

I missed this while working on yesterday's message.

Let's see below:


> Okay! :-)
> 
> About the use of ECN, I agree as you say there should only be a few fragments
> and not increasing might be okay. However, you would need to clarify that the
> window is reset for each new datagram, I guess, right? 

Agreed. Let's change appendix C, more below;

> Also I don’t think you
> necessarily need to reduce to 1 on CE marking but maybe halve the window or
> something. Or you leave this open like “If an E flag is received the window
> SHOULD be reduced, at least by 1 and at max to 1. Halving the window for
> each E flag received, could be a good compromise but needs further
> experimentation.”…

Thanks for this text! Appendix C now becomes:
"
   Congestion on the forward path can also be indicated by an Explicit
   Congestion Notification (ECN) mechanism.  Though whether and how ECN
   [RFC3168] is carried out over the LoWPAN is out of scope, this draft
   provides a way for the destination endpoint to echo an ECN indication
   back to the fragmenting endpoint in an acknowledgment message as
   represented in Figure 4 in Section 5.2.  While the support of echoing
   the ECN at the reassembling endpoint is mandatory, this specification
   only provides a minimalistic behaviour on the fragmenting endpoint.
   If an E flag is received the window SHOULD be reduced, at least by 1
   and at max to 1.  Halving the window for each E flag received, could
   be a good compromise but needs further experimentation.  A very
   simple implementation may just reset the window to 1 so the fragments
   are sent and acknowledged one by one.  Note that the action on ECN
   only applies till the end of the datagram and the next datagram uses
   the configured Window_Size again.

"

I'd like to fully converge before I publish again

> I wonder if it would be good to say a bit more about the recommended values
> for the window size, as I think 32 will usually in most network not limit
> transmission (and the limiting value will be IFG) while with a size of 3, that's
> very conservative to not overload the network (and will be slow than the limits
> induced by IFG). Is my understanding correct?

I'd believe so:

Note that the exact use of the ack and the window_size is left to implementation. The optimistic one could send all the fragments up to window_size and ask for an ack only on the last one, wait for the bitmap, which means a gap of RTT/2, and resend the losses.
Then again we want to leave room for learning. The pessimistic implementation could set the bit on the first fragment to check that the path works and open the window only upon receiving the ack. It could then set an ack bit again on the second fragment and then use the window as a credit to send up to window_size before it is blocked. In that case, if the ack comes back before the window starves, the gating factor is the IFG. If the ack does not arrive in time, the window_size is the gating factor.

If the sender uses the window_size as a credit, the window of 3 will be the gating factor that starves the sender and causes gaps longer than IFG as soon as you have more than 3 hops in a TSCH network and 5-6 hops in a single frequency mesh. I guess that's what you mean by slower. The more hops the more it will hurt.

I initially used nb_of_hops as a rule of a thumb. That was an approximation of RTT/(XMIT+IFG) in the case one fragment needs to progress only to the second hop before the next can be sent (case of TSCH). 

The assumptions behind are that an rfrag_ack takes the same path and the same time as the rfrag, which is mostly correct. So if we spread the fragments with IFG, keeping busy one hop out of 2 on both directions means aligning the window_size of the nb of hops. As you pointed out this is not a congestion control measure, it is just the window that is not "slower" than IFG, per your expression. The text that uses RTT/(XMIT+IFG) being equivalent I'm happy either way. Because knowing the RTT is the same problem as knowing the number of hops so we do not need to indicate both. 

3 is when you h	ave no idea of either RTT or nb of hops. it is conservative and will starve the TSCH sender if we have more than 3 hops. It also protects the intermediate node that can be very constrained, to at most 3 fragments per datagram that it relays. Note that I'm not aware of any full duplex radio available in LLNs (though there are prototypes for 5G). So this is really a corner case.

As written 32 indicates the last fragment of the datagram, which is usually way less than 32. When used, as you understood well, the only protection is IFG, and I'm must say I'm quite happy with the progress we made on that text together.

Please let me know, should I change something more?

Pascal